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ABSTRACT 


An  algorithm  has  been  developed  which  uses  the  complementary  slackness  principle 
to  take  completely  arbitrary  primal  and  dual  solutions  to  a  linear  program  with 
doubly-bounded  variables  Into  the  optimal  solutions.  The  algorithm  Is  not  a 
new  method;  viewed  In  the  proper  context,  It  can  be  thought  of  either  as  an 
elaboration  of  the  primal-dual,  composite,  breakpoint- tracing,  or  complementary 
pivot  algorithms:  as  an  extension  of  the  out-of-kllter  or  black-box  methods  for 
network  floWs;  or,  finally,  even  as  a  special  way  of  looking  at  the  original 
simplex  algorithm.  However,  It  possesses  certain  pedagogical  advantages: 

1.  Proper  emphasis  Is  placed  on  complementary  slackness  as  the  fundamental 
constructive  principle  of  linear  programming,  and  Infeaslblllty  and 
unboundedness  are  related  to  secondary  roles. 

2.  Arbitrary  starting  solutions  are  allowed,  and  arbitrary  lower  and 
upper  bounds  on  the  variables  are  handled  naturally. 

3.  One  activity  at  a  time  Is  "worked  on;"  complementary  slackness  always 
Indicates  what  operations  are  necessary;  no  artificial  distinction  Is 
made  between  "real,"  "artificial,"  or  "slack"  variables. 

A.  The  imbedded  linear  program  Is  of  an  extremely  simple  type,  which 
reveals  the  essential  nature  of  simplifications  which  can  be  made  in 
models  of  special  structure. 

5.  Very  few  set-theoretic  proofs  and  tableaux  rules  are  needed,  almost 
all  operations  being  described  on  the  optimality  diagram  for  each 
activity. 

Almost  all  of  the  simpler  procedures,  such  as  Phase  I,  the  dual  simplex  method, 
parametric  programming,  the  primal-dual  algorithm,  etc.  can  be  viewed  as  special 
cases  of  the  complex  algorithm  which  use  special  starting  solutions  and  special 
heuristics,  r 
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COMPLEX: 


A  COMPLEMENTARY  SLACKNESS,  OUT-OF-KILTER 
ALGORITHM  FOR  LINEAR  PROGRAMMING 
by 

William  S.  Jewell 


0.  INTRODUCTION 

The  purpose  of  this  paper  is  to  present  a  complementary  slackness,  out-of¬ 
kilter  algorithm  for  the  following  dual  linear  programs: 


(0.1) 


(0.2) 


n 


Minimize 


c-  i  c.x, 

j-i  3  3 


ih  ^ ' b* 


lj -xi -ki 


m  n  n 

Maximize  0  -  £  b  y  -  £  k  u  +  J  1  w 

i-1  x  x  J-l  3  3  J-l  3  3 


y^  unrestricted 

uj -  0 


m 


Jj  yi*i3  '  a]  +  v3  ’  cj 


(i  *  1,2,  . . . |  m) 
(j  -  1,2,  n) 


where  no  restriction  is  made  on  the  constants  except  l ^  for  all  j  .  (For 
a  symmetric  form,  see  Section  9.) 

The  algorithm  to  be  presented  is  not  a  new  method;  viewed  in  the  proper  light 
it  can  be  thought  of  either  as  an  elaboration  of  the  primal-dual,  composite,  break- 
polrt  tracing,  or  complementary  pivot  algorithms;  as  an  extension  of  the  out-of- 
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kilter  or  black-box  methods  for  network  flows;  or  finally,  even  as  a  special  way 
of  looking  at  the  original  simplex  algorithm.  In  fact,  each  of  these  methods  can 
be  produced  by  the  use  of  appropriate  heuristics  within  the  procedure  presented  in 
Section  6. 

Why,  then,  another  algorithm?  When  teaching  linear  programming,  one  is  struck  • 

by  the  fact  that  almost  all  "advanced"  topics  are  merely  repetitions  of  the  same 
basic  concepts  of  pivoting  (to  move  from  one  extreme  point  to  another)  and  pricing- 
out  (to  determine  a  desirable  direction  in  which  to  move),  with  slightly"  different 
emphasis  on  which  variable  is  to  be  increased  or  decreased,  and  by  how  much  that  and 
other  variables  are  allowed  to  change.  What  is  needed,  it  seems,  is  a  general 
algorithmic  framework  in  which  all  the  various  extreme-point  procedures  can  be 
explained  as  variants  of  a  common  procedure  which  use  special  heuristics. 

In  the  author's  opinion,  such  a  corrnon  simplex  procedure  should  emphasize  the  » 

following  points: 


(a)  Much  greater  emphasis  should  be  placed  on  the  complementary- 
elackness  relationship  as  the  fundamental  working  principle 
of  programming;  questions  of  feasibility  and  boundedness 
should  be  relegated  to  a  secondary  role; 

(b)  The  simplex  procedure  is  a  local ,  or  incremental  move  in  which 
one  basis  change  is  made  to  Improve  some  functional;  the  prob- 

•  lem  is  to  know  how  to  make  this  one  move  and  Interpret  the 
results — the  remaining  steps  will  always  "look"  Identical; 

(c)  Arbitrary  starting  solutions  should  be  allowed,  and  tedious 
conventions  on  signs  of  variables  and  constants,  form  of  con¬ 
straints,  "real"  versus  "artificial"  variables,  etc.  should 
be  eliminated  as  much  as  possible; 

(d)  Less  emphasis  needs  to  be  given  to  tableaux  and  a  variety 
of  formal  rules  for  row  and  column  manipulations;  the 
algorithm  and  the  current  "state"  of  the  solution  should 
remind  one  of  the  correct  rule  to  be  used. 


Given  that  the  elements  for  such  a  development  have  been  available  since  1959 
[7,8,11],  it  is  remarkable  how  many  variants  and  elaborations  have  been  presented 
(often  with  intriguing  prior  arguments  for  efficiency),  but  how  little  unifying  and 
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synthesizing  work  has  been  done.  We  do  not  pretend  that  the  algorithm  presented 
here  will  be  a  definitive  one,  but  only  hope  that  it  will  begin  to  reveal  the 
underlying  coherence  and  harmony  between  the  various  approaches,  and  how  simple 
and  straightforward  a  general  theory  is. 

The  report  is  in  the  following  main  groupings:  the  first  four  sections 
examine  the  central  ideas  of  the  optimality  diagram  and  the  incremental  linear 
program  subroutine;  Sections  5-7  present  the  main  algorithm  and  its  proofs; 
Section  8  presents  some  of  the  many  options  available  in  using  the  algorithm, 
followed  by  a  discussion  of  symmetric  formulations  in  Section  9;  Section  10  dis¬ 
cusses  the  important  special  case  of  network  flow  problems;  and  Sections  11-13 
conclude  with  a  review  of  the  basic  ideas  of  the  report,  an  appreciation  of 
J.  B.  Dennis'  important  work,  and  a  survey  of  the  extensions  which  are  possible. 
The  Appendices  present  certain  fine  points,  such  as  the  organization  of  tableaux, 
and  the  interpretation  of  classical  algorithms  as  special  cases  of  the  common 
procedure. 


THE  OPTIMALITY  PRINCIPLE  AND  DIAGRAM 


To  fulfill  point  (0.3a),  we  begin  by  defining  the  constructive  principle  which 

will  be  used  to  solve  (0.1)  and  (0.2)  and  express  this  principle  as  a  set  of  n 
2 

diagrams  in  (R  ,  one  for  each  activity. 

Until  Section  9,  we  assume: 

every  eet  of  values  of  the  primal  variables  x  -  x°  (j  -  1,2 . n) 

(1.1)  l  o 

will  always  satisfy  the  equality  constraints  \  aijxj  "  ^  . m) 

either  by  adjoining  slack  variables  of  any  sign  in  the  formulation,  or  by  adjoining 
error  variables  of  any  sign  after  the  Initial  solution  is  chosen.  These  particular 
constraints  will  never  be  violated  during  the  algorithm  and  can  henceforth  be  Ignored 
however,  x°  may  exceed  either  of  its  bounds  or  l ^  . 

Let 


m 

(1.2)  *i  ■  \  yj®i .  (J  “  1»2»  ...»  n) 

3  i-1  3 

be  the  profitability  of  activity  j  .  For  given  initial  values  |x°;y°|  of  the 
primal  and  dual  variables  satisfying  (1.1),  we  define  the  state  of  activity  j  as 
the  pair  of  values  (xj'zj)  »  an<*  Petition  the  state  space  as  shown  in  (1.3). 


of  Activity  j 
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For  convenience,  let 

(1.4)  if  -  if  or  B“  or  K-  j  U  -  L  or  B  or  K  ;  U+  -  L+  or  B+  or  K+  . 

The  state  a  current  solution  at  iteration  t  (t  -  0,1,2,  ...)  can 

be  shown  most  clearly  on  an  optimality  diagram  (x  ;z  )  for  each  activity  j  , 

J  J 

shown  in  Figure  1.1;  for  example,  U  consists  of  the  solid  horizontal  and  vertical 

lines.  Some  of  the  states  may  be  missing  in  degenerate  cases. 

« 

The  following  is  just  a  restatement  of  what  Is  usually  called  the  weak  theorem 
of  complementary  slackness: 


(1.5) 


Optimality  Principle 

For  a  set  of  values  j  »  satisfying  ][  a^x^  “  b^ 

(i  -  1,  ...»  m)  to  be  optimal,  it  ie  necessary  and  sufficient 
that  J  c  U  for  all  activities  J  ■  1,  ...»  n  . 


(The  strong  theorem  of  complementary  slackness  states  that  there  Is  at  least  * 
one  optimal  solution  which  does  not  have  any  activities  on  the  corner  points , 

(B  and  Xj  ■  lj)  ,  (B  and  x^  -  k^)  ;  however,  we  shall  not  need  this  fact  in  the 
sequel. ) 

An  activity  in  U  will  be  called  conforming  [14],  or  in-kilter  [9];  a  "non-U" 
activity  in  U  or  U+  Is  nonoon forming,  or  out-of-kilter. 

The  basic  Idea  of  the  algorithm  to  be  presented  In  Section  6  Is  as  follows: 

The  Initial  solution  determines  the  stste  of  all  activities.  An  arbitrary  non- 
conforming  activity  Is  selected,  and  changes  in  the  variables  are  made  by  a  sub¬ 
routine  to  make  this  selected  activity  more  conforming  (in  a  sense  to  be  made 
precise  in  Section  7);  these  changes  leave  all  currently  conforming  activities 
ln-kllter,  and  no  unselected  nonconforming  activity  becomes  more  nonconforming. 

The  primary  pedagogical  advantage  of  the  optimality  diagram  is  that  all  features 
of  the  algorithm  can  be  explained  directly  on  the  diagram;  this  simplifies  notation 
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2 .  THE  INCREMENTAL  LINEAR  PROGRAM 


The  key  procedure  in  the  algorithm  is  a  subroutine  which  determines  the 
incremental  changes  to  be  made  from  the  current  solution.  If  8}  are  the  values 


th 


determined  during  the  t  (t  ■  0,1,2,  ...)  iteration,  the  incremental  displace- 
ment ,  of  activity  j  during  the  next  iteration  is 


(2.1) 


Ej  ’  XJ  '  *3 


j  ■  1 . n 


The  subroutine  consists  of  the  following  incremental  linear  program: 


Maximize  A  -  +£ 

-  s 


(2.2) 


j-i  Vj 


Minimize 


\i  -  5j  -  kj 


1  Vj  '  W 


(l  m  1,2,  ...,  m) 

(J  ■  1 » 2 ,  ...,  n) 


(2.3) 


k  v«  -  “j  +  -j  ■  l 


0  J^8 


+1  j-8 


U j  i  0  »  “ j  i  0 
x\±  unrestricted 

The  selected  activity  index ,  s  ,  will  be  furnished  to  the  subroutine,  as  will  an 
Independent  set  of  column  vectors,  J)  ,  from  the  matrix  (a^)  •  The  values  of 
the  bounds  provided  will  always  be  such  that 


(2.4) 


—  j<  Aj  <  0  <  < 


( j  m  1.2 ,  ...,  n) 


So  that  the  incremental  origin  is  always  (bi»sic)  feae.'ble 


For  the  selected  activity: 


(2.5) 


if  the  problem  is  to  maximize  £#  *  A#  ■  0  ;  0  <  <_  +»  ; 

if  the  problem  is  to  minimize  £  ,  <  A  <  0  ;  k  -0. 

8  —  8  8 


The  optimality  diagram  for  the  incremental  program, 


(2.6) 


versus 


» 


for  activity  j  is  shown  in  Figure  2.1.  Note  that  some  activities  may  have  part 
or  all  of  the  c^-axis  or  the  C^-axis  as  conforming  states. 

It  follows  from  (2.4)  and  (2.5)  that: 


The  values  £^  ■  0  ,  J  e  ,  constitute  a  basic  feasible  solution 

to  (2.2)  with  ■  0  ,  (i  -  1,2 . m)  and  -  0  , 

(2.7)  (J  -  1,2,  . ..,  n)  as  corresponding  solutions  to  (2).  (As  usual , 
"0  •  J  /£  )•  Furthermore ,  this  basis  and  these  values  can¬ 
not  be  optimal  solutions  to  (2.2)  and  (2.3). 


The  initial  Incremental  program  is,  in  fact,  in  complementary  pivot  form 
(Section  9B).  There  are  four  possibilities  for  the  optimal  solutions, 

M  •  M  *  M ! 


(a) 

The  optimal  value  of 

*s 

,  * 

is  unbounded  (£^  ■  +•»  or 

— ) 

(b) 

The  optimal  value  of 

*8 

reaches  its  nonzero  bound 

(£*  ■  k  or  A  )  . 

(2.8) 

(c) 

The  optimal  value  of 

*s 

is  nonzero,  and  the  displacement 

of  some  other  activity  reaches  a  bound  (0  <  £  <  k 

or 

X8  <  *8  <  0  *  aiUl 

* 

* 

or  £j  ■  Kj  for  some  j 

4  s) 

(d) 

The  optimal  value  of 

*s 

is  zero,  but  th*  given  *9 

is 

not  the*  index  set  of 

the 

optimal  basin. 

FIGURE  2.1  (a)  and  (b):  OPTIMALITY  DIAGRAMS  FOR  THE  INCREMENTAL  PROGRAM 
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Appendix  A  discusses  ways  in  which  this  special  linear  program  can  be  organized 
in  tableau  form;  Section  8  discusses  simplifications  which  can  be  used  for 
problems  of  special  structure,  and  the  possibility  of  selecting  several  activities. 
In  all  cases,  except  (2.8a),  we  know  from  the  optimality  diagrams  in  Figure  2.1: 


The  finite  optimal  solutions  to  (2.2)  and  (2.3)  satisfy : 


*  *  r 

If  x<  <  <  *4  •  then  C*  “  l 


(2.9) 


.  (  °  j*>) 


*  OO 

then  ’■i  -  <y  • 


Obvious  modifications  apply  when 
Infinite. 


0  ,  or  one  or  both  of  these  is 


In  particular,  we  note  that  not  both  £*  and  c*  can  be  simultaneously 
zero.  In  addition  to  providing  the  optimal  displacement  of  the  primal  variables, 
we  shall  see  in  Section  4  that  the  incremental  subroutine  also  furnishes  the 
optimal  gradient  of  dual  displacement. 


3.  ACTIVITY  SELECTION  FOR  THE  INCREMENTAL  LINEAR  PROGRAM 

The  selected  activity,  s  ,  which  is  to  be  "worked  upon"  in  the  incremental 
subroutine  can  be  chosen  from  among  any  of  the  nonconforming  activities  which  vio¬ 
late  the  Optimality  Principle  (1.5).  Various  heuristic  procedures  for  this 
selection  are  discussed  in  Section  8. 

For  "primal-infeasible"  activities,  it  is  clear  that  the  rule 


(3.1) 


s  e  L  ,  B  ,  or  K  —  maximize  £  ; 

8 

•f*  +  *f 

s  e  L  ,  B  ,  or  K  —  minimize  £  , 

8 


will  move  the  state  of  activity  s  chosen  towards  feasibility.  However,  this  rule 
is  also  correct  if  l  <  x  <  k  and  8  i  B  ,  as  we  shall  see  in  the  next  section. 

8  —  S  —  S 

How  far  should  the  {£ ^ }  be  allowed  to  move?  Clearly,  a  conforming  activity 
should  not  be  allowed  to  leave  U  .  Or,  conversely,  if  some  j  t  U  ,  the 
incremental  movement  should  be  stopped  when  the  activity  reaches  U  .  Finally,  for 
finiteness,  it  is  desirable  to  prevent  "non-U"  activities  from  becoming  more  so  by  * 
moving  counter  to  the  rule  (3.1). 

This  leads  to  the  following  rules  for  specifying  the  bounds  for  the 

(t+l)8t  application  of  the  subroutine,  in  terms  of  the  current  values  |x*| 

(t  **  0,1,2,  •••)  . 


SET: 


(3.2) 


J  e  L" 
JcB“,k" 
JtL.K 
J  e  B 

]  i  L*  ,  B* 
i  E  K+ 


-<*;  -  v 
-<*]  - 
-<xj  •  v 


*d  ■  xj 
kj  -  xj‘ 


k  —  x 

j  1 


( j  “  •••*  n) 
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The  directions  of  change  and  the  allowed  maximal  increments  are  conveniently 
summarized  on  the  Optimality  Diagram  in  Figure  3.1.  There  are  no  infinite  dis 
placements  allowed  in  the  diagram  as  shown!  but  if  ■  06  »  or  for 

j  ,  it  would  be  possible  to  obtain  the  unbounded  solution  of  (2.8a). 


FIGURE  3.1:  OPTIMALITY  DIAGRAM  FOR  ACTIVITY  j  ,  SHOWING  DIRECTION  AND 

MAXIMAL  PRIMAL  DISPLACEMENTS  ALLOWED  FROM  DIFFERENT  STATES  (•) 
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4.  BREAKPOINT  STEPPING  WITH  THE  INCREMENTAL  LINEAR  PROGRAM 


From  the  above  discussion,  the  new  activity  levels  ■  x*  +  £  obtained. 

J  J  J 

by  adding  the  optimal  Incremental  values  from  the  subroutine  to  the  old  activity 
levels  obviously  keep  all  conforming  activities  so,  and  do  not  move  nonconforming 
points  farther  away  from  or  through  the  optimality  diagram  (see  Section  7  for  a 
discussion  of  conformity  measures).  Furthermore,  the  primal  displacement  only 
permits  the  following  limiting  changes  of  state  from  iteration  t  to  t  +  1  : 


(a) 

l" 

L 

(d) 

L+  -►  L 

(4.1) 

(b) 

b“ 

-*•  B 

(e) 

B+  B 

(c) 

k" 

-*•  K 

(f) 

K+  K 

f  j  i  U 

lat  iteration 


0 


or 


<g>  (*;  <  k,) 

<h)  (*;  >  t}) 


,  t+l  t. 

lx,  ■  k. 


J 

■r 


3) 


J  c  B 

at  iteration 


J 


At  least  one  such  change  will  occur,  possibly  at  a  zero-change  level. 

Hopefully,  the  selected  activity  s  might  undergo  a  change  of  type  (a)  -  (f), 
but  in  general,  £s  may  only  move  partway  toward  a  conforming  state,  as  shown  by 
the  horizontal  line  in  Figure  4.2.  In  fact,  from  (2.8d)  it  is  possible  that  C*  -  0  . 

After  these  horizontal  (primal)  movements  are  made  on  the  various  diagrams, 
the  dual  solution  to  the  incremental  linear  program  fumishee  the  appropriate 
gradient  for  vertical  (dual)  changes  through  the  formulae : 

(4.2)  y^+1  -  y*  +  Bn*  (i  -  1,2 . m) 

(4.3)  zj+1  -  z*  +  64*  (j  -  1,2 . n) 

» 

and  appropriate  selection  of  the  gradient  step  size,  0  . 


Consider  Figure  4.1.  If  the  same  requirements  are  placed  on  dual  displacements 
as  for  the  primal  ones  (all  conforming  activities  remain  so,  and  displacement  stops 


when  a  nonconforming  activity  becomes  conforming),  then  it  follows  from  (2.9)  and 
Figure  2.1  that  after  the  primal  change  (4.3)  we  have: 


(4.4) 


IF: 

THEN: 

* 

'i 

0 

(a) 

JeL  ,B“,  or  (K  and  x*+*<£^) 

* 

<j±° 

unlimited 

t 

(b) 

Je(K  and  tj<_  xj^ck^) 

or  K 

If 

* 

A 

o 

*rci 

0  must  be  * 

"CJ 

(c) 

Je(L+  and  Xj+1>^  )  ,B+, 

or  K+ 

* 

unlimited 

t 

(d) 

Je(L+  and  £j<x*+*<  k^) 

or  L 

If 

o 

A 

*  *n 

c  -z 

0  must  be  <  ■ 

(e) 

JeB  and  £.<x*+^<k 

3  J  J 

O 

1 

K  T-) 
k* 

unlimited 

(f) 

JeB  and 

s-° 

unlimited 

(g) 

JeB  and  x*+*-kj 

unlimited 

for  every  activity  J  .  Changes  which  do  not  limit  0  are  shown  as  dotted  lines 
in  Figure  4.1.  (In  Section  7  it  is  shown  that  these  displacements  are  also  re¬ 
ducing  nonconformity  in  a  certain  sense.) 

We  conclude  that  all  conditions  (4.4)  are  satisfied  for  all  activities  for 
every  value  of  step  size: 


By  selecting  0  -  0*  ,  we  guarantee  that  at  least  one  change  of  state 


FIGURE  4.1:  OPTIMALITY  DIAGRAM  FOR  ACTIVITY  j  ,  SHOWING  DIRECTION  AND 
MAXIMAL  DUAL  DISPLACEMENTS  ALLOWED  FOR  DIFFERENT  STATES  (•) 


‘W/n 
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(A. 6) 


(a)  K  ►  B 

(c)  L  ->  B 

(b)  K~  -*•  B 

(d)  L+  -+  B 

occurs.  If  s  becomes  conforming,  then  all  the  £^  may  be  zero,  in  which  case 

* 

6  can  be  set  to  zero.  If  the  states  referred  to  in  (4.5)  are  nonexistent,  and 

* 

s  is  nonconforming,  then  0  =  00  ,  and  the  dual  solution  is  unbounded. 

Of  particular  interest  is  the  trajectory  traced  out  by  (x  ,z  )  as  it  is 

S  8 

"worked  upon"  by  the  incremental  linear  program.  Suppose  first  that  (x°,z°)  e  if 
.  s  s 


as  shown  in  Figure  4.2;  then  the  subroutine  will  maximize  £ 


s 


The  result  of 


*  * 

this  subroutine  will  be  A  *»  £  ,  creating  a  nonnegative  displacement  towards  the 

s 

right  to  (x*,z*)  ,  possibly  all  the  way  to  state  K  ,  (or  unbounded  if  tc  *  “)  . 

8  8  8 

(For  z°  <  c  ,  the  first  step  might,  of  course,  be  stopped  by  entry  into  state  L  .) 

S  8 

*  * 

Then  calculation  of  the  dual  changes  will  change  z  by  the  amount  0  £ 

s  s 

But,  from  (2.9)  and  (3.2),  if  the  horizontal  segment  does  not  reach  to  state  K 

*  * 

or  L  ,  then  £  <  <  ,  and  £  -  -1  .  In  other  words,  the  dual  change  moves  the 

8  8  8 


A  1  1  * 

trajectory '  dawnuarda  by  a  positive  amount  6  to  the  point  (x  ,z  )  .  If  0  «•  »•  , 

8  S 

then  the  trajectory  moves  downwards  off  the  diagram,  showing  dual  unboundedness 

(i.e.,  primal  infeaslblllty — there  no  basis  change  at  any  price  which  will  bring 

x  up  to  l)  . 

8  8 

The  net  result  is  that,  after  successive  applications  of  the  subroutine,  the 
trajectory  of  (xt,zt)  describes  what  Dennis  [8]  calls  the  breakpoint  curve ,  a 
sequence  of  nonnegative  horizontal  segments  and  positive  vertical  segments  which 
leads  either  to  unboundedness  or  infeasibility,  or  to  an  intersection  with  one  of 
the  conforming  "U"  states.  Similar  remarks,  "in  reverse",  apply  to  the  trajectory 
traced  out  by  some  activity  in  U+  ,  for  which  the  subroutine  would  minimize  £ 

8 

(See  also  Section  8  for  further  possibilities.) 

Unselected  non-conforming  activities  also  follow  a  breakpoint  curve  at  each 
application  of  (2.2)  and  (4.2)  until  conforming,  or  selected. 
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5.  GENERAL  OUTLINE  OF  THE  ALGORITHM 

The  general  outline  of  the  algorithm  should  now  be  apparent.  Starting  with 
an  arbitrary  solution  to  the  primal  and  dual,  the  states  of  each  activity  are 
identified,  using  (1.3)  and  Figure  1.1.  An  arbitrary  nonconforming  activity,  s  , 
is  selected  and  worked  upon,  using  the  incremental  subroutine.  By  following  the 
breakpoint  curve.  Figure  (4.2)  either  unboundedness  or  infeasibility  is  shown,  or 
s  is  put  in-kilter.  Then  another  nonconforming  activity  is  selected,  and  so  on, 
until  all  activities  are  in  U  ,  and  an  optimal  solution  is  obtained. 

There  are  still  several  points  to  be  cleared  up,  such  as  the  finiteness  of 
the  procedure,  both  with  the  regard  to  the  subroutine,  and  the  main  algorithm 
(Section  7).  In  addition,  there  are  various  options  available  at  each  iteration 
which  can  be  used  in  devising  various  heuristic  procedures;  these  will  be 
discussed  in  Section  8  and  when  comparing  the  algorithm  with  others  in  Appendix  D. 

We  now  present  the  main  algorithm. 
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6.  COMPLEX:  A  COMPLEMENTARY  SLACKNESS.  OUT-OF-KILTER  ALGORITHM  FOR  LINEAR 
PROGRAMMING 


(6.1) 


0.  Select  arbitrary  values  of  x^  ■  x°  (j  -  1,2,  ...,  n) 

satisfying  (1.1)  and  ■  y°  (1  -  1,2,  ...,  m)  .  Select  an 
arbitrary  set  of  m  independent  columns  for  the  initial  set 
«9°  (Appendix  B) .  Set  t  ■  0  . 

1.  Identify  current  states  (x*;z*)  of  activities  as 

u‘-  l"  ,  B"  ,  or  K‘  i  U  -  L  ,  B  ,  or  K  i  U+  ■  L+  ,  B+  ,  or  K+  . 
If  all  j  c  U  ,  the  current  solution  is  optimal. 

2.  Otherwise,  select  an  arbitrary  nonconforming  state  s  . 

Solve  the  Incremental  Linear  Program 

n 

Maximize  A  ■  \  c 

j-1  3  3 


n 


,1  Vi  - 0 


*} 


n 

Minimise  £  (k.u.  - 

j«l  J  J  J  J 

ViJ  -  “J  +  “J  '  -‘i 

u  j  i  0  ,  Wj  >_  0 

r\±  unrestricted 


(i  m  1,2,  ...,  n) 
(j  ■  1,2,  . . . ,  n) 


with  initial  basis  ;  starting  solution  ■  0 

(J  m  1»  •••»  n)  , 


-'r 


(6.2) 


("J^ 

1+1  j  ■  8  e  U 

)°r 

y-1  j  -  s  e  U 


and  defined  by  (3.2). 

3.  If  A*  »  +»  ,  the  primal  problem  (0.1)  is  unbounded. 


A.  Otherwise,  set 


(6.3) 


(6.  A) 


t+1  t  ,  * 

*1  ’  xj  +  ?'J 


(j  “  1»  •  • • »  n) 


Find  the  step  size,  0  ,  from  (A. 5)  and  the  related 


discussion. 


If  e*  »  •  t  the  primal  problem  (0.1)  is  infeasible. 


Otherwise,  set 


t+1  t  *  * 


(i  ■  1,2,  ...,  m) 


(6.5) 


t  *  * 

zi  +  6  'j  ’ 


(J  “  1»2 ,  • •  • ,  n) 


(retain  the  current  optimal  basis  as  the  starting  basis  S 

for  the  next  iteration)  and  repeat  Step  1. 


JPMWW*****1*1"  ■ 
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7.  CONVERGENCE  AND  FINITENESS 

Assume  temporarily  that  the  current  solution  is  "primal-feasible",  i.e., 
Xj  kj  .  By  direct  substitution,  the  difference  between  the  primal  and 
dual  functionals  is: 


(7.1) 


&  "C-Bm  £  u  (k  -  x  )  +  l  w  (x  -  l  ) 

J  J  J  jBl  J  «J  J 


which  will  be  called  the  total  deviation  of  the  current  solution. 

In  terms  of  the  optimality  diagrams,  the  total  deviation  is  just  the  sum  of 
the  individual  activity  deviations t  which  are  the  shaded  areas  shown  in  Figure  7.1. 

Furthermore,  this  result  does  not  require  or  to  be  finite,  provided 

one  uses  the  usual  "transfinite  algebra"  in  which  the  infinite  bound  is  replaced 
by  a  number  +M  which  will  be  considered  larger  than  any  number  to  which  it  is 
compared  during  the  calculations.  Thus  to  a  point  like  (g)  in  Figure  7.1,  an 
area  (M  -  x^)  would  be  contributed  to  the  total  deviation  2)  .  This  is  just 
the  pricing-in  term  which  would  be  added  to  force  the  point  out  of  the  "dual- 
infeasible"  region  above  the  line  zj  “  cj 

A  similar  device  can  be  used  to  make  arbitrary  points  "primal-feasible"  as 
well.  For  example,  if  the  current  x^  >  k^  or  <  ,  we  may  consider  that  x^ 

is  really  an  unbounded  activity  with  piecewise  linear  cost  structure 


(7.2) 


cost  of  activity  j 


Cj'j  +  M(tj  ■  V 

Vxj 


-  -  <xj  < 

kj 


Ykj  +H<xj  ‘  V  k3  '  xj  -  “  ■ 


which  gives  the  transfinite  extensions  to  the  optimality  curves  shown  in  Figure 
7.2  (see  also  Chapter  VI,  j'jj). 
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Direct  calculation  of  JO  =  C- 3  for  the  extended  problem  (7.2)  leads  again 
to  the  conclusion  that  the  area  between  the  point  (x^jz^)  and  the  corresponding 
diagram  is  the  deviation  due  to  activity  j  ;  as  shown  in  Figure  7.2,  this 
deviation  may  be  transfinlte  only  ,  or  may  have  both  finite  and  transflnlte 

regions  . 

Thus,  as  a  given  activity  follows  its  breakpoint  curve,  as  shown  in  Figure  7.3, 
it  follows  from  the  algorithm  that: 


Every  nonzero  horizontal  or  vertical  displacement  permitted 
by  the  algorithm  gives  a  finite  decrease  to  the  total  deviation 
0.3)£>  =  C-&t  equal  to  the  decrease  in  shaded  area  between  the 
points  (XjjZj)  and  the  optimality  curves,  summed  over  all 
activities. 


In  classical  terminology,  the  displacement  from  I  to  II  in  Figure  7.3 

makes  activity  J  "primal-feasible",  and  from  II  to  III,  "optimal"  (if  k  was 

J  » 

+«  ,  all  vertical  displacements  reduced  "dual-infeasibility",  as  well);  however, 
in  our  extended  definitions,  all  horizontal  displacements  decrease  C  ,  and  all 
vertical  displacements  increase  6  . 

Since  the  only  displacements  allowed  decrease  JO  ,  and  since  each  non- 
conforming  activity  is  in  the  subroutine  until  it  becomes  conforming,  it  is  clear 
that  the  algorithm  converges. 

* 

The  only  possible  source  of  degeneracy  occurs  in  the  incremental  subroutine, 
where  cycling  can  be  avoided  by  the  usual  perturbation  or  lexicographic  techniques [A] . 
Even  if  the  maximal  value  of  A  is  zero.  (2.8),  the  decrease  in  j V  will  be  positive, 
and  a  new  basis  will  be  selected.  Thus  the  algorithm  is  finite. 

A 

An  alternate  "finite"  proof  that  an  infinite  number  of  steps  with  8  finite 
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and  with  A  “0  cannot  occur  is  as  follows: 

£ 

(a)  Since  A  -  0  t  no  changes  of  the  type 

(i)  K+  -*•  K 

(ii)  K+  -*•  K 

(iii)  L_  -*•  L 

(iv)  L+  ■*>  L 

(v)  (j  e  B)  (Xj  -  ij)  *  (Xj  >  £j) 

(vi)  (j  e  B)  (Xj  <  kj)  ^  (xj  *  kj) 

(7.  A) 

can  occur. 

(b)  If  an  activity  moves  from  L  •>  B  or  K  -*■  B  during  one 
dual  change,  its  index  will  enter  the  basic  sgt  J)  of 
the  subroutine  and  remain  there  as  long  as  A  “  0  . 

(c)  Otherwise,  0*  is  determined  by  a  transition  L+  -*•  B 
or  K~  -*■  B  decreasing  the  number  of  activities  which 
enter  into  (A. 5). 

(d)  Since  there  are  a  finite  number  of  activities,  there 
can  be  gnly  a  finite  number  of  dual  changes  until  either 
some  A  is  finite  ("breakthrough") ,  or  the  set  in 
(4.5)  is  empty  (unbounded  dual). 
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8.  ALTERNATIVES  FOR  THE  INCREMENTAL  PROGRAM  SUBROUTINE 

The  incremental  linear  program  (2.2)  can,-  of  course,  be  solved  by  any  method 
available;  a  fairly  compact  tableau  method  which  utilizes  its  homogeneous,  double- 
bounded  structure  is  given  in  Appendix  A.  In  this  section,  we  indicate  several 
alternative  approaches  which  may  be  used. 

A.  Working  on  Several  Nonconforming  Activities  at  the  Same  Time 

If,  in  fact,  a  general  simplex  tableau  procedure,  as  outlined  in  Appendix  A, 
Is  being  used  to  solve  the  incremental  program,  then  one  may  wish  to  do  more  than 
maximize  (or  minimize)  the  selected  nonconforming  activity.  In  particular,  one 


may  work  on  all  such 

activities  at 

once, 

by  setting 

• 

f+1 

If 

J  c  if 

(8.1) 

c.  -  (-1 

If 

J  c  U+ 

0  otherwise 


Instead  of  (A.  1)  and  (6.2).  This  does  not  complicate  the  subroutine  outlined  In 
Appendix  A  and  may  make  several  activities  conforming  In  one  application  (with 
possibly  more  pivot  steps)  of  the  subroutine. 

Or,  one  may  select  some  of  the  nonconforming  activities  to  work  on— for  example 
the  activities  with  transfinite  deviations  may  be  worked  on  first,  as  In  the  usual 
"Phase  I"  procedures  (Appendix  D) . 

Finally,  the  magnitude  of  the  coefficients  Is  immaterial  to  convergence 

of  the  algorithm,  and  one  may  choose  to  put  more  or  less  pressure  on  certain  non¬ 
conforming  activities,  based  on  some  heuristic  choice;  this  is  the  basis  for  most 
proposals  which  combine  "Phase  I"  and  "Phase  II". 


B.  Primal-Freezing  Nonconforming  Activities  for  Single-Step  Subroutines 


simple  as  possible.  One  way  to  do  this  is  to  "freeze"  all  nonconforming  variables, 
other  than  the  one  selected,  at  their  current  values  by  setting 

(8.2)  Xj  -  Kj  -  0  J  i  U  +  {s} 

Instead  of  following  (3.2).  X  or  k  are  set  as  usual. 

8  8 

In  this  way,  as  the  subroutine  (Appendix  A)  attempts  to  increase  (or  decrease) 
£  ,  only  two  bounded  possibilities  occur: 

(a)  C  reaches  one  of  its  own  nonzero  bounds;  the 

8 

subroutine  terminates  with  s  conforming  and  the 

current  basis  J  unchanged  (but  with  possibly 

* 

different  ,  J  c  J)  ;  all  are  un¬ 

changed  (equal  to  Cj)  . 

(8.3)  or 

(b)  One  (or  more)  basic  ^  reaches  a  bound;  s 

replaces  1  in  the  basis  by  making  a  pivot  on 
some  a.  (A. 6).  The  subroutine  terminates  after 

*  *  l>\ 

one  pxvot  with  +  c£  0  ,  an<*  other 

C*  +  e.  -  0  ,  J  e  J1  +  (s)  . 

^ .  J  J 

(The  usual  remarks  about  ties  apply.) 

In  this  simpler,  but  more  restricted  procedure,  it  is  clear  that  no  con- 

« 

trol  is  maintained  over  the  sign  of  ,  J  t  U  +  {s>  .  Thus,  other  nonconforming 
activities  may  become  more  so  (i.e.,  their  deviations  may  increase)  at  the  dual¬ 
changing  step  which  follows.  This  gives  some  theoretical  problems  in  convergence, 
but  in  most  cases,  one  can  show  that  one  of  the  functionals  is  moving  in  the 


correct  direction. 
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If  the  resulting  value  of  E  from  (8.3)  were  then  such  that  x  were 
"primal-feasible",  then  it  would  be  desirable  to  choose  6  without  regard  to  the 
other  nonconforming  activities.  This  would  then  make  s  conforming  in  a  single 
pass  through  the  algorithmt  although  other  activities  might  "overshoot" — i.e., 
conforming  ones  might  become  nonconforming  (see  D  below) . 

The  above  procedure  is  used  in  the  primal  simplex  algorithm.  If  several 
variables  are  brought  in  at  once,  the  procedure  is  called  "block-pivoting". 


C.  Dual-Freezing  Certain  Nonconforming  Activities 


In  a  similar  way,  one  can  dual-freeze  up  to  m  nonconforming  variables  Cj 
by  requiring  that  their  Indices  be  in  any  basis  J  of  the  Incremental  subroutine. 
This  restriction  then  means  that  the  bounds  and  (3.2)  for  these 

variables  cannot  be  effective ,  and  the  corresponding  may  have  arbitrary  sign; 

thus  these  other  nonconforming  variables  may  have  increasing  deviations. 

If  an  attempt  is  made  to  make  activity  s  conforming  in  one  step,  this  may 


drive  some  x  ,  j  c  B  nonconforming.  Thus,  this  possibility  is  usually  followed 
in  reverse;  i.e.,  some  ,  J  c  B  (;^  -  0)  is  moved  towards  its  bound  until  some 
positively  or  negatively  priced  (s  reaches  the  appropriate  bound.  This  "row- 
pivoting"  is  the  procedure  used  in  the  dual  simplex  algorithm  (Appendix  D). 


D.  Overshooting  Conforming  Limits 

In  general,  the  limits  on  the  Cj  end  0Cj  ,  J  i  U  ,  have  been  chosen  so 

that: 

(a)  no  conforming  activity  passes  through  a  conforming  state  and 
then  becomes  nonconforming  again; 

(b)  no  previously  conforming  activity  becomes  nonconforming. 

As  we  have  seen  above,  however,  when  working  on  a  particular  activity,  or 

e 

set  of  activities,  it  may  be  desirable  to  get  this  activity  conforming  "at  all 
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costs".  This  may  conceptually  be  very  bad  if  violating  (3.2)  or  (A. 5)  makes  more 
activities  nonconforming.  However,  in  certain  special  cases,  we  may  be  able  to 
argue  convergence  on  one  of  the  functionals. 

E.  Special  Structure  Models 

Actually,  the  incremental  subroutine,  as  we  see  it,  is  a  method  for  choosing 
the  direction  of  the  vector  which  will  maximize  the  rate  of  increase  of  A  , 

subject  to  a£jSj  *  0  *  with  certain  directions  "frozen";  the  actual  maximum  dis¬ 
placement  in  either  primal  or  dual  is  of  secondary  importance. 

In  problems  of  special  structure,  it  may  be  possible  to  follow  through  the 
effect  of  changing  one  variable  on  the  other  variables  explicitly;  then  the  various 
operations  of  "pivoting"  can  be  carried  out  in  sequential  form,  without  continual 
reduction  of  the  matrix  to  get  the  current  trade-off  coefficients  . 

The  most  common  example  of  this  kind  occurs  in  network  flow  models,  where  the 
allowed  changes  in  the  (5 j )  correspond  to  an  incremental  increase  in  arc  flows 

» 

around  a  loop  Including  arc  s  .  (Section  10.) 

F.  Dual-Stepping 

Nothing  in  the  algorithm  should  be  construed  so  as  to  give  a  special  place  to 
the  primal  problem.  One  can  just  as  well  define  the  {n^}  as  the  absolute  dis¬ 
placements  of  the  (y . }  ,  and  work  on  a  selected  n  through  a  homogeneous  dual 
in  the  Incremental  subroutine.  For  example,  the  pivoting  procedure  may  be  clearer 
in  the  transposed  matrix  (a^)  • 

In  this  case,  nonnegative  vertical  steps  in  Figure  4.2  are  taken  first, 
followed  by  positive  horizontal  displacements,  since  all  degeneracy  (at  the  corner 
points  of  Figure  1.1)  arise  in  the  subroutine.  Thus,  "row  pivots"  become  the 
natural  changes,  and  "column  pivots"  would  require  looking  ahead  to  the  "primal-" 
changing  step  (6.4)  and  (6.5). 
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G.  Bound-Tightening 

When  a  given  activity  is  made  more  conforming,  it  is  possible  some  other 
nonconforming  variable  may  move  coherently.  If  the  incremental  subroutine  makes 
several  basis  changes,  it  may  be  worthwhile  to  "tighten"  the  bounds  on  these  other 
variables  as  the  pivoting  progresses  to  prevent  them  from  "slipping  back". 
Actually,  except  for  didactic  examples,  this  possibility  is  quite  rare. 

H.  STARTING  SOLUTIONS 

In  many  problems,  special  starting  solutions,  basic  or  not,  may  be  available. 
If  these  are  felt  to  be  "reasonable",  or  near  optimal,  they  certainly  should  be 
used  in  place  of  a  completely  arbitrary  solution.  On  the  other  hand,  if  one  had 
previously  found  a  basic  set  J)°  ,  and  the  related  inverse  basis  in  (a^j)  ,  then 
one  should  use  the  corresponding  basic  solution,  feasible  or  not,  solely  in  the 
interest  of  efficiency. 

I.  ARGUMENTS  FOR  EFFICIENCY 

It  should  be  clear  from  the  discussion  of  this  Section  and  Appendix  D  that 
any  prior  arguments  for  efficiency  of  a  certain  heuristic,  particularly  those 
based  on  whether  one  is  moving  "inside",  "outside"  or  "on"  a  certain  convex  poly¬ 
tope,  are  doomed  to  failure. 

The  COMPLEX  algorithm  takes  any  starting  solution,  basic  or  not,  feasible  or 
not,  and  converts  it  into  an  extreme  point  by  adding  artificial  bounds  (2.4). 

Thus  all  starting  solutions  "look  alike",  in  a  certain  sense,  and  progress  in  the 
same  manner  as  a  basic  feasible  solution  would  move  over  the  original  polytope. 

One  would  have  to  make  extensive  numerical  trials  for  special  classes  of  problems 
in  order  to  clearly  demonstrate  the  superiority  of  one  heuristic  over  another. 

Most  such  experiments  have  concentrated  on  how  to  select  a  i:  ^conforming  activity 
when  using  the  heuristic  described  in  Section  8B  above,  and  starting  with  basic 


feasible  solutions. 
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9.  SYMMETRIC  FORMULATIONS 

The  problem  (0.1)  (0.2)  has  been  stated  as  an  equality  primal  with  doubly- 
bounded  variables,  since  this  is  often  the  formulation  in  real  problems.  On  the 
other  hand,  certain  models,  such  as  two-person  games,  look  more  natural  in  a 
symmetric  primal-dual  format;  this  approach  is  often  favored  for  aesthetic  reasons, 
as  well.  In  this  section,  we  modify  the  algorithm  of  Section  6  to  a  symmetric  form. 

Consider: 


(9.1) 


(9.2) 


n 


Minimize  C=  £  c.x 
J-l  J  J 


X  auxj  -  bi 


XJ  -  0 


A 

Maximize  u  ■  £  b.y 

i-1  1  1 


y±  1° 

X  yiau  - c] 


By  defining  nonnegative  slack  variables: 


(9.3)  . 


’  }l  Vi  *  bi  - 


(i  *  1,2,  ...,  m) 
(j  "  1»2,  . . . ,  n) 


(i  -  1,2,  . . . ,  m) 


(9.4) 


•j  ■  c3  -  Vu  i 0  «  * 1*2 . n) 


the  constraints  in  (9.1)  and  (9.2)  are  changed  to  equalities;  since  they  will 
remain  equalities  during  the  algorithm,  they  are  henceforth  ignored,  and  all 
attention  is  focussed  on  the  extended  variables: 
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1 

[\ 

(k 

“  1,2, 

•  •  •  9 

n) 

(9.5) 

m  i 

I 

rk-n 

(k 

■  n+1, 

n+2. 

•  •  •  9 

n+m) 

r  s. 

(k 

=  1,2, 

•  •  •  » 

n) 

(9.6) 

|# 

1  k 

ifk 

[yk-n 

(k 

*=  n+1, 

n+2 , 

•  •  •  9 

n+m) 

where  k 

runs  over  the 

range  (1,2, 

•  •  •  » 

n;n+l. 

n+2 , 

•  •  •  » 

n+m)  to  take  in  the 

appropriate  real  or  slack  variables.  The  extended  constraint  matrix  of  (9.1) 
consists  of  (a^)  augmented  by  an  m  x  m  negative  identity  matrix: 

(9.7)  (o^)  -  ((a^),'  -I) 

The  restatement  of  the  Optimality  Principle  (1.5)  is: 


(9.8) 


Optimality  Principle 

A  feasible  solution  of  nonnegative  values  Kti  * 
optimal  if  and  only  if 

.  V  ft  - 0 


%8 


for  all  k  ■  1,2,  ...»  m+n  . 

The  optimality  diagrams  corresponding  to  (9.7)  are  shown  in  Figure  9.1.  Note 
that  this  figure  is  reversed  and  normalized  from  Figure  1.1.  Thus,  the  breakpoint 
trajectories  will  have  reversed  "dual"  changes,  increasing  (decreasing)  from  left 
to  right  (right  to  left).  Vn  keep  the  same  states  L  ,  L  ,  L+  ,  B  ,  B  ,  and 
K  as  before  lor  the  current  extended  state 
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For  completeness,  we  restate  the  algor  ithm._of  Section  6  in  symmetric  form; 
some  details  on  the  extended  tableau  are  given  in  Appendix  C.  No  proofs  of  the 
algorithm  need  be  given,  since  the  extended  problem  is  exactly  in  the  form  (0.1) 
and  (0.2).  However,  some  "complementary  pivot"  interpretations  are  given  in  sub¬ 
section  B. 


A.  Symmetric  Form  of  the  Complex  Algorithm 


0. 


1. 


2. 


Select  arbitrary  values  of  x^  -  x^  (j  -  1,2 . n)  and 

y^  ■  y°  (i  -  1,2,  ....  m)  and  use  (9.3) (9. A)  to  calculate 


the  remaining  components  of  {*;}  -  • 

Select  an  arbitrary  set  of  m  independent  columns  Jl°  ,  in 
the  extended  constraint  matrix,  (a^)  (say,  the  negative 
identity  matrix  of  the  last  m  columns). 

Set  t  -  0  , 


,  or  K  ;  U  «  L  or  B  ;  U+  -  L+ 


Identify  the  current  states 
U"  -  L"  ,  B 
k  c  U  ,  the  current  solution  is  optimal. 
Otherwise,  solve  the  Incremental  Linear  Program: 


all  variables  as 
If  all 


(9.9) 


Maximize  A 


m+n 

ek*k 

k-1 


m+n 

I  flik^k  "  ® 
k-1  * 


Xk  -  5k  -  Kk 


(i  —  1,2,  •••,  m) 

(k  -  1,2 . m+n) 


Minimize 


m+n 

J,  <w  -  W 

k-1 

1  Wk  -  \  +  \  ■  -ck 

(i  *  1,2 ,  • • • ,  m) 

(k  -  1,2 ,  . . . ,  m+n) 

nA  unrestricted 

with  initial  basis  J)*  ,  and  starting  solution  "  0 
(k  ■  1,2 ,  ...»  m+n)  . 

The  coefficients  in  the  functional  have  arbitrary  value 
and  sign: 


c 


k 


'  =  0 

£  0 

.10 


keU 
k  e  U 


selected  to  work  on  one  or  several  nonconforming  variables  at 
the  same  time 

The  bounds  are: 


If  A  ■  +•  ,  the  primal  problem  (9.1)  is  unbounded. 


Otherwise,  set: 
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(9.13) 


/t+1 


+  < 


(k  -  1,2 ,  ...»  m+n) 


(9.14) 


5.  Find  the  step  size  6  from 


0  "  min 

k 


[tt/Ji  “ k) 


(9.15) 


6. 


7. 


where  only  indices  k  are  allowed  for  which:  1.  0  ; 

the  denominator  is  nonzero;  and  the  numerator  and 
denominator  are  of  the  same  sign. 

If  the  set  of  indices  in  (9.14)  is  empty,  0  ■  «°  ,  and 

the  primal  problem  (9.1)  is  infeasible. 

Otherwise,  set: 

k  “^k  ”  6  ^i^ik  (k  -  1,2 . m+n) 


and  repeat  Step  1. 


B.  Complementary  Pivot  Methods 

Appendix  C  points  out  how  certain  of  the  basis  changes  in  the  extended 
matrix  (<t^)  can  be  interpreted  as  "dual  pivots",  in  the  sense  of  the  row  operations 
of  the  dual  simplex  method  (Appendix  D.A).  This  symmetrlzation  is  formalized  in  the 
complementary  pivot  methode  of  Cottle  [31,  Dantzig  [6J,  and  Lemke  [12),  which  were 
developed  for  a  larger  class  of  problems  (Section  13) . 

Instead  of  the  extended  variables  (9.5) (9.6),  attention  is  focussed  on  the 
variables: 

(k  ■  1,2,  ...,  n) 

(k  -  n+1 ,  n+2 ,  . . . ,  n+m) 

(k  ■  1,2,  ...,  n) 

(k  -  n+1 ,  n+2 ,  . .  , ,  n+m) 


(9.16) 


(9.  17) 


-(*k 


W  - 

k 


yk-n 

®k 

rk-i« 


AO 


This  has  the  effect  of  reversing  the  axes  on  the  last  m  optimality  diagrams  of 

Figure  9.1,  thus  making  the  "real"  dual  variables  y  abscissae,  together  with 

i 

the  "real"  primal  variables  .  The  problem  (9.1)  (9.2)  is  also  usually  stated 
as  a  combined  (irrt-n)  x  (nrfn)  primal-dual  problem,  together  with  feasibility 
requirements  which  are  identical  with  the  optimality  requirements  (9.8). 

In  terms  of  our  model,  complementary  pivot  theory  stresses  the  case  when 
exactly  one  pair  of  complementary  variables,  say  *n  (9.16)  (9.17)  is 

nonconforming;  this  implies  exactly  one  other  pair  is  at  the  LB  corner  point,  say 
2  ■  id  “  0  (possibly  more  than  one  if  there  is  degeneracy).  This  point  e  is 

then  moved  into  either  L  or  B  in  a  manner  to  reduce  the  nonconformity  of  the 

variable  h  ,  some  other  variable  4  then  moving  into  its  corner.  If  4  moves 

into  the  corner  on  B  ,  it  leaves  on  L  at  the  next  iteration,  and  vice  versa; 

this  is  what  is  meant  by  complementary  pivoting,  and  is  a  natural  observation  from 
the  Incremental  subroutine  of  Appendix  C.  The  procedure,  of  course,  terminates, 
when  variable  h  becomes  conforming. 

Actually,  starting  with  just  one  nonconforming  solution  pair  (z^;  Wh^  *s 
quite  difficult,  in  general,  and  the  only  starting  solution  methods  proposed  [5] 
seem  to  require  introduction  of  artificial  variables.  Following  the  effect  of  this 
proposal  through  in  terms  of  Figure  9.1  reveals: 

(a)  All  initial  and  subsequent  solutions  are  in  L  ,  B  ,  or  B  , 

(b)  The  nonconforming  polnt(s)  which  is (are)  currently  farthest 
away  (in  a  linear  sense)  are  worked  upon. 

(c)  Primal-  and  dual-freezing  are  used  as  needed  to  keep  all 
solutions  in  the  above  states. 

These  special  rules  then  reduce  the  complementary  pivot  procedure  to  a  combination 
of  the  primal- simplex  and  dual-simplex  procedures,  that  is,  a  co..  >osite  method 
(Appendix  D.IJ* 


<  0  and 


-°). 


However,  we  have  previously  seen  (2.7)  that  the  incremental  program ,  (2.2) 

or  (9.9),  is  already  in  complementary  pivot  form,  since  only  £  »  0  is  noncon- 

6 

forming  at  the  beginning  of  the  Incremental  subroutine.  Thus,  the  COMPLEX  approach 
essentially  reduces  every  initial  solution  to  a  related  complementary  pivot  problem. 
In  addition,  the  nuisance  problems  of  infeasibility  and  unboundedness  are  handled 
separately. 


C.  A  Doubly  Double-Bounded  Symmetric  Problem 

As  an  ultimate  symmetric  variant,  the  reader  may  wish  to  try  rewriting  the 
algorithm  of  subsection  A  for  the  following  doubly  double-bounded  eyrrmetric  problem : 

n  m 

Min Cm  l  c.x  -  l  [d  .max  (0  ,r .  )  -  e.max(0,-r . )  ] 

•j-1  J  J  i-1  1  1  1  1 


(9.18) 


(9.19) 


*uxj  -  ri  -  bi 

kj 

(i  -  1,2,  ...»  m) 
(J  ■  1*2,  •  •  • ,  n) 

m  n 

Max  -  \  +  l  [ljinax(0,Sj)  -  k^max(0,-s^)  ] 

di  i  ^  *i 

X  yi*u + -j  -  cj 


whose  optimality  diagrams  are  given  in  Figure  9.2  a  and  b 


D.  General  Piecewlsc-Linrar  Convex  Costs 

It  is  also  appropriate  to  remark  that  the  COMPLEX  algorithm  can  easily  be 
extended  to  general  piecewise-linear  convex  costs.  The  resulting  optimality  dia 
grams  would  then  have  many  horizontal  and  vertical  segments;  only  a  few  details 
in  the  algorithm  would  need  to  be  changed.  (Sec  also  Section  12  and  [8].) 


ITT  DIAGRAMS  FOR  DOUBLY  DOUBLE-BOUNDED  SYMMETRIC  PROBLEM 
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10.  NETWORK  FLOW  MODELS 

A  special  class  of  problems  of  great  practical  interest  are  the  network 
flow  models  [9,10,14]  for  which  the  constraint  matrix  is  the  node-arc  incidence 
matrix,  giving  either  Kirchoff's  Current  or  Voltage  Laws  in  the  primal  or  dual. 

For  these  models,  the  homogeneous  incremental  program  takes  on  the  following 
simple  form: 

(a)  Find  a  loop  (cycle)  of  special  arcs  in  the  network  that 
can  take  an  incremental  amount  of  flow; 

(10.1)  or 

(b)  Establish  a  set  of  potentials  on  each  node,  so  that  the 
sum  of  potential  differences  around  any  loop  of  special 
arcs  is  zero. 

Either  or  both  of  these  steps  is  handled  in  netowrk  problems  by  means  of  a  simple 
labeling  technique,  which  is  merely  a  way  of  "unraveling"  *  fects  of  t  pivot 

change;  by  changing  the  flow  variables  around  a  loop  (adding  an  incremental 
circulation  flow),  one  increases  or  decreases  only  the  variables  necessai;  to  keep 
the  flow  conservation  laws  satisfied.  (If  the  problem  is  stated  in  single  source- 
sink  form,  the  pivoting  procedure  may  find  a  How- augmenting  path.)  As  usual, 
determining  the  actual  bound  on  the  increment  of  flow,  or  determining  the  changes 
lr.  potential  to  "break  down"  a  new  loop,  are  secondary  calculations  which  can  be 
performed  independently.  Thus,  the  calculations  proceed  from  a  basic  solution  (a 
tree  of  arcs) ,  adding  a  new  vector  (a  cotree  arc  which  forms  a  unique  loop  with  the 
tree),  through  a  pivot  change  (determining  an  arc  in  the  loop  to  be  removed),  to 
the  new  basis  (another  tree). 

Viewed  in  terms  of  the  optimality  diagram  and  the  algorithm  of  Section  6,  the 
various  methods  can  be  viewed  as  follows: 

(a)  The  stepping-stone  methods  are  primal  simplex  algorithms  which 

always  maintain  bap^v  primal- feasible  solutions  as  shown  in 
Figure  D.l.  A  set  of  potentialsf  placed  on  the  basic  tree,  '\.kes 
a  cotree  arc  nonconforming.  Then,  flow  is  rerouted  to  deLeii..ine 
a  new  basic  solution,  and  so  on  until  optimal. 


(b)  The  Hungarian- Ford- Fulkerson  methods  were  responsible  for  the 
development  of  the  general  primal-dual  method  for  linear 
programs  discussed  in  Appendix  D  and  F.  Usually  all  non¬ 
conformity  is  placed  on  the  return  arc  from  sink  to  source , 
which  starts  with  zero  flow.  Thus,  incrementally  least-cost 
flow-augmenting  paths  are  sought  until  all  flow  is  allocated. 

(c)  The  out-of-kilter  method  is  exactly  the  algorithm  of  Section  6. 

Actually,  after  the  important  early  papers  of  Ford  and  Fulkerson  see  [9,14]  for 
references)  the  state-of-the-art  was  such  that  many  people  realized  that  the 
complementary  slackness  conditions  were  the  key  to  dealing  with  arbitrary  initial 
condition  (Appendix  A  of  [10]).  In  fact,  an  independent  development  of  the  out-of- 
kllter  approach  may  be  found  in  the  1958  thesis  of  J.  B.  Dennis  [8].  The  emphasis 
Is  on  electrical  analogues,  and  elementary  activities  (Section  12),  but  his 
"black-box"  approach,  and  "breakpoint-tracing"  are  identical  to  our  notion  of  a 
selected  activity,  and  sequential  primal-dual  changes  needed  to  force  this  activity 
to  be  conforming.  This  Important  paper  treats  general  programming  problems  in 
this  same  light  (Sections  12  and  13). 

One  can  also  extend  the  simplified  pivoting  of  ordinary  network  flows  to 
problems  where  there  are  arbitrary  multipliers  of  flow  on  each  arc  [10a],  giving 
arbitrary  coefficients  in  the  node-arc  incidence  matrix.  However,  here  the 
appropriate  basis  is  not  a  tree,  and  the  appropriate  "unwound"  form  of  a  pivot 
step  is  to  find  a  floo-absorbing  loop  in  the  network.  Extensions  to  multi-com- 
commodlty  flows  have  also  been  suggested  (see,  for  example,  [10b]), 
efficiency  of  the  algorithms  is  less  satisfactory. 


but  here  the 
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11.  THE  SIMPLEX  METHOD  AND  ITS  HEURISTICS 


By  this  point,  our  central  thesis  should  be  evident: 

all  the  extreme  point  methods  of  linear  progranvning  are 
equivalent  to  the  original  simplex  mcthodt 
modified  by  the  following  heuristic  choices: 

(a)  How  is  the  problem  formulated — inequalities,  equalities,  symmetric 
form,  self-dual,  upper  or  lower  bounds,  etc..? 

(b)  What  initial  solution,  if  any,  is  available  for  the  primal  and/or 
dual? 

(c)  Is  the  initial  solution  basic,  or  is  a  natural  starting  basis 
available? 

(d)  Which  group  of  nonconforming  variables  is  to  be  worked  on  first? 

Is  any  subset  to  be  worked  on  simultaneously,  or  will  they  be 
handled  individually?  What  criterion  of  nonconformity  selects 
among  these? 

(e)  What  is  the  attitude  towards  the  other  nonconforming  variables? 

Are  they  to  be  bounded  towards  conformity?  Primal-  or  dual-frozen? 
Ignored? 

(f)  Is  a  general  simplex  (pivot)  subroutine  to  be  used,  or  a  special  al¬ 
gorithm?  Will  it  do  primal  steps  first,  or  dual  steps? 

(g)  Are  single-pivot  changes  to  conforming  states  to  be  made?  How  much 
primal-  or  dual-change  overshooting  is  to  be  allowed? 

The  primary  advantage  of  the  algorithm  presented  here  is  a  pedagogical  one, 
that  different  variants  are  easily  presented,  and  compared.  Also,  the  problem 
does  not  need  to  be  forced  into  any  artificial  format;  construction  of  the  ap¬ 
propriate  optimality  diagram  always  "reminds"  one  of  the  appropriate  procedure 
to  be  followed. 

Finally,  the  COMPLEX  algorithm  presents  a  general  framework  within  which  new 
heuristics  can  be  explored;  certainly  the  basic  similarity  of  the  various  break¬ 
point  curves  suggests  that  questions  of  relative  efficiency  must  be  decided  com¬ 
putationally,  and  not  on  the  basis  of  whether  a  solution  stays  feasible  in  a 
certain  sense,  or  always  mr^'es  some  activity  conforming  at  each  step.  In  this  way, 
we  distinguish  between  the  theory  and  the  art  of  optimization. 


12.  BREAKPOINT-TRACING  ELEMENTARY  ACTIVITIES  AS  BUILDING  BLOCKS  AND  THE 

BREAKPOINT- THEORY  ALGORITHMS  OF  J.  B.  DENNIS 

In  this  section  we  will  consider  the  "construction"  of  an  arbitrary  linear 
program  ftom  certain  elementary  activities.  In  this  way,  we  hope  to  shed  some 
light  on  the  pioneering  work  of  J.  B.  Dennis  [8]. 

As  one  analyzes  the  mechanism  of  the  simplex  algorithms,  it  becomes  clear 
that  there  are  only  a  certain  number  of  elementary  operations  performed,  and 
that  each  regular  activity  could  be  thought  of  as  a  composite  of  simpler 
elements.  Typically,  we  have:  a  "pricing"  mechanism  which  turns  an  activity 
"on"  or  "off";  a  primal  (in)equality ;  and  a  dual  (in)equality . 

The  three  canonical  elementary  activities  might  be: 


unrestricted 


To  build  up  composite  activities,  we  use  the  following  construction  rules 
between  two  arbitrary  elements  with  given  optimality  diagrams,  (x^;z^)  and 
,  to  construct  a  new  activity  (xo;zq)  • 

(a)  " Series "  Combination 


\  ‘  *1  ’  x2  8  zo  ■  21  +  *2 


(b)  " Parallel "  Combination 


(12.2) 


"o  ■  X1  +  x2  1  *.  ■  ‘l  ■  *2 


plus  the  unary  operation: 

(c)  Trane formation 


xo  •  Kxl  !  xo  ■  K21 


In  this  way,  compound  activities  with  very  general ’btaircase"  diagrams  can  be  built 
up.  For  example,  the  compound  activity  shown  in  Figure  12.2  (a)  can  be  built  up  out 
of  the  five  elementary  activities  shown  in  Figures  12.2  (b)  -  (f)  by  adding  the 
three  constraints 


(12.3) 


0  "  “  *2  +  x3 


0  -  -x. 


-  x3  +  x4 


and  substituting  xq  in  whatever  other  constraints  hold  for  the  compound  activity. 

Rearrangements  of  (12.3)  suggest  the  various  ways  in  which  the  "assembly"  of 

(x  •  z  )  can  occur. 

O’  o 


I 


Such  an  expanded  representation  naturally  leads  to  many  more  elementary 
activities  than  real  ones.  But,  conversely,  the  various  simplex  operations 
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degenerate  considerably  for  the  types  (12.1),  and  it  may  be  possible  to  devise 
special  algorithms  for  hand  calculations,  particularly  when  the  problem  is 
"weakly  connected". 

The  use  of  elementary  activities  is  nothing  more  than  a  faithful  transcription 
of  the  idealized  current  elements  (diode,  voltage  source,  current  source)  in 
J.  B.  Dennis’  1958  thesis  [8].  The  notion  of  breakpoint  stepping  and  operating 
with  the  optimality  diagrams  are  also  his;  and,  from  a  certain  viewpoint,  this 
entire  paper  might  be  said  to  be  a  natural  interpretation  of  his  method. 

But,  for  some  reason  which  this  author  does  not  understand,  very  little 
attention  has  been  paid  to  this  fundamental  paper;  in  the  next  section  we  shall 
see  that  Dennis'  breakpoint-stepping  ideas  can  also  be  used  in  a  larger  context. 


nop mu 
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13.  EXTENSIONS 


Wolfe  [16]  has  shown  how  quadratic  programs  can  be  calculated  by  applying 
complementary  slackness  conditions  to  an  enlarged  problem  in  which  the  primal  and 
dual  variables  are  "coupled"  by  linear  relationships.  The  basic  outline  of  an  out- 
of-kilter  method  for  these  problems  was  given  by  Dennis  [8],  who  introduced  the 
idealized  elementary  activity  which  has  a  straight  line  ^k  "  Rk^k  °Ptimallty 
diagram  (a  "resistor");  there  appears  to  be  no  difficulty  in  extending  this 
approach  in  the  directions  we  have  outlined  here.  Recently,  Lemke  [12]  has  shown 
that  bimatrix  games  can  also  be  put  in  complementary  pivot  form,  which  would  then 
represent  another  extension  for  arbitrary  starting  solutions.  In  both  these  cases, 
the  breakpoint  curve  consists  of  straight  lines  at  arbitrary  angles,  as  well  as 
horizontal  and  vertical  segments,  and  the  appropriate  theory  may  be  found  in  [8], 
Chapters  5  and  6. 

In  principle,  it  would  also  be  possible  to  extend  these  ideas  to  general 
convex  programming,  provided  that  one  knew  how  to  move  both  primal  and  dual 
solutions  simultaneously  along  the  monotone  optimality  curves  defined  by  the 
Legendre  transformation  which  gives  the  dual  problem.  For  this  reason,  nonlinear 
solution  methods  appear  to  depend  more  strongly  on  special  Initial  conditions  than 
linear  methods  do.  Another  well-exploited  procedure  is  to  use  quadratic 
programming  as  a  local  approximation  (Chapter  7  of  [8]). 
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APPENDIX  A.  ORGANIZING  THE  INCRF, MENTAL  PROGRAM  TABLEAU 

Because  the  incremental  program  subroutine  (2.2)  and  (2.3)  is  in  a  special 
form,  in  this  section  we  consider  how  calculations  might  be  carried  out  in  tableau 
format.  Naturally,  to  develop  an  efficient  procedure  for  computer  calculations, 
■any  other  factors,  such  as  size  of  storage,  round-off  error,  matrix  sparseness, 
etc.  must  be  considered.  As  discussed  in  Section  8E,  special  algorithms  may  also 
be  available  for  problems  with  special  structure.  However,  the  procedure  outlined 
below  is  adequate  for  hand  calculations  and  as  a  starting  point  for  planning  com¬ 
puter  organization. 

There  are  two  novel  features  of  the  incremental  program: 

(a)  it  is  homogeneous; 

and  (b)  its  variables  are  doubly  bounded. 

The  first  fact  gives  an  easy  starting  solution  in  all  cases;  there  is  also  no 
right-hand  side  to  keep  track  of.  To  use  (b)  efficiently,  the  upper  and  lower 
bounding  relationships  are  calculated  outside  the  main  tableau,  and  we  keep 
separate  track  of  the  current  values  of  the  "nonbasic"  variables  which  are  at 
their  upper  or  lower  bounds. 

A  suggested  format,  as  it  might  appear  at  the  start  of  the  first  Iteration  is 
shown  in  Figure  Al.  The  coefficients  (a^)  are  plac®d  in  the  main  m  x  n  array. 
In  the  next  row  above  are  placed  the  "pricing"  terms,  +  cj  »  w*t*1 

is  to  be  maximized 
is  to  be  minimized 


(Al) 


’+  1  if  5 

l 

-  1  if  C 


‘j-° 


J  4  8 


But  see  Section  8A  for  other  possibilities. 


Since  the  current  values  of  the  variables  are  "lost"  during  an  upper-  (and  lower-) 
bounding  technique,  the  { C j )  are  displayed,  together  with  the  current  values  of 
the  {Kj}  and  {X^}  ,  in  three  rows  above  the  pricing  terms. 

For  completeness,  an  m  x  m  unit  matrix  is  adjoined  to  the  matrix  of  coeffi¬ 
cients  so  that  the  dual  variables  { }  and  the  inverse  of  the  current  basic  are 
obtained  automatically.  In  a  computer  code,  a  more  efficient  scheme  such  as  the 
product  form  of  the  inverse  would  be  used. 

Once  the  initial  basic  set  of  variables,  i  =  J  ,  is  specified,  then  the 
tableau  can  be  reduced  to  the  normal  form  shown  in  Figure  A2  by  the  usual  Gauss- 
Jordan  reduction  procedure,  and  the  relative  prices  }  reduced  to  zero  for 

j  e  J  .  This  simultaneously  gives  the  inverse  of  the  current  basis  and  dual 
variables  for  the  incremental  program.  Pivoting  operations  are  carried  out  in  each 
application  of  the  subroutine  using  the  rules  below;  the  last  form  of  the  tableau 
in  a  given  iteration  can  be  used  to  begin  the  next  application  of  the  subroutine 
(Appendix  B) .  , 

In  a  doubly-bounded  procedure,  all  basic  variables  satisfy: 

(A2)  5  J  Cj  +  Gj  “  °  J  e  J  * 

The  nonbaslc  variables  are  at  their  upper  or  lower  bound,  and,  at  optimality  (2.9): 


If  Cj  ♦  >  0 

if  tj  ♦«,«<> 


{j  ’  V 

■  xj 


j  l&. 


Nonbasic  Activities 


Thus , 


Rule  for  Selection  of  a  Candidate  to  Enter  the  Basis 
Scan  the  Pricing  Row  for  Some  Nonhasic  Activity  eiA  such  that 


(A4) 

or 

(a) 

;  +  e 

e  e 

>  0 

and 

£  <  < 
e  e 

(b) 

C  +  e 
e  e 

<  0 

and 

£  >  A  . 

e  e 

(i)  If  none 

exist. 

the 

current 

Uj)  ,  {n±}  and 

(Cj.)  are  optimal 

(ii)  Otherwise,  £  is  a  candidate  to  enter  the  basis. 

e 

Now,  if  some  £  is  to  enter  the  basis  from  its  lower  bound,  a  positive  [negative] 
e 

trade-off  coefficient,  a^e  ,  in  row  i  ,  column  e  of  the  tableau  Indicates  that 
the  basic  variable  corresponding  to  row  i  (i.e.,  the  unit  matrix  has  an 

entry  in  column  l  ,  row  i  )  must  decrease  [increase]:  the  bound  ^ 

may  thus  limit  the  increase  of  .  as  will  its  own  bound,  ,  Similar  remarks 

apply  if  £e  is  to  enter  the  basis  from  its  upper  bound. 

The  smallest  such  change  determines  the  actual  change  to  be  made  in  £e  : 
if  it  is  a  basic  variable  which  reaches  a  bound,  replaces 

in  the  basis,  with  a  as  pivot ;  if  £  goes  all  the  way  to  its  other  bound, 
the  basic  set  of  variables  remains  unchanged,  and  only  the  values  of  the 
are  changed.  Obvious  remarks  apply  to  ties  and  unboundedness. 

Thus  follows: 


Rule  for  Selection  of  Candidate  to  Leave  the  Basis ,  and  for  Changing 
the  Values  of  the  Incremental  Variables 
1.  For  a  given  candidate  activity  e  ,  a  nonzero  trade-off  coefficient  °ie  * 

(A3)  and  the  corresponding  basic  activity  £(i)  ,  partition  A  into  the  following 
sets,  and  define  the  indicated  displacements ,  6^  ,  for  all  ]  c  J  +  {e} 


j 


r 


A .  6 


(A6) 


4 


r,  +  c  >0  and  a .  <  0 

e  c  ie 


(a)  £(t)  c  J  iff  Sor 


lC  +  c  <0  and  a.  >  0 
e  e  ie 


rC  4  e  >0  and  a.  >  0 
1  e  e  ie 


(b)  t(i)  c J  iff  (or 


U  +  e  <0  and  a,  <  0 
e  e  ie 


(c)  Otherwise  £(i)  e  J)  -  :  6^  =  « 


’  5£  C  ' 

,<vV/<+“  i.). 

K-v'<+v>) 

;«rV 

(d)  Define  6  .  jj”  . 

e  e 


if  £  4  e  >  0 

e  e 

if  C  4  e  <  0 
e  e 


(A7) 


2.  Compute  the  allowed  displacement }  6  . 


6  =  min  )  6  ;  min  6.;  min  6. 

i  6  iJ  £  *  ■-  £ 


£e«fl 


(a)  If  6  =  °°  ,  the  incremental  program  is  unbounded.  (Return  to  main 

* 

program  with  A  *>  00  .) 

(b)  If  the  minimum  6  in  (A7)  is  computed  from  /  or  J  ,  then 
activity  e  enters  the  basis  at  level  6  <  6e  ,  driving  some  basic 
activity  £  to  its  lower  or  upper  bound,  whence  it  leaves  the  basis. 

Pivot  on  the  appropriate  uig  and  reduce  all  rows  of  the  tableau  plus 

.» 

the  pricing  row  to  normal  form  with  the  new  basis  «*  ■  +  {e}-  {£} 

(c)  If  6  is  determined  by  6^  ,  then  goes  to  a  bound,  and  the 

t 

current  basis  »3  remains  unchanged.  Jl  *  J)  . 

(d)  In  case  of  a  tie,  the  choice  is  immaterial  (avoid  a  pivot  operation, 
if  possible). 


I 
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3.  Compute  the  new  values  of  the  {^}  : 


«.  I-  <) if  |e;  ♦ <  o 


-  s  +  6  lf  J  £  J 

= 5 i  - 6  «  1 c  * 


if  J  t  J-  J+  -  J' 


Then,  a  new  candidate  is  selected,  using  (A4)  ,  and  the  steps  are  repeated 
until  the  optimality  criterion  (A3)  is  satisfied.  Control  is  then  returned  to  the 
main  routine,  with  the  optimal  values  {£*}  ,  (n*)  ,  and  U*}  from  the  appropriate 

j  j 

*  r.  * 

points  of  the  tableau,  and  A  "  A  Ej  ‘ 

The  next  call  for  the  subroutine  will  clear  {?  },  ....  and  give  new  values  to 
{X.}  {*.}  and  { c . )  ;  however,  the  rest  of  the  tableau  can  be  saved  in  its  last 

J  j  j  *  ■ 

(reduced)  form,  with  the  optimal  basis  from  this  iteration  (Appendix  B). 
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APPENDIX  B.  SELECTING  THE  INITIAL  AND  SUBSEQUENT  BASIS 

To  get  the  incremental  program  subroutine  started,  it  is  necessary  to  specify 
an  initial  basic  set  *3  °  .  However,  since  *  0  for  all  variables  at  the 
beginning  of  each  of  the  subroutines,  any  selection  of  m-independent  columns  of 
(a^j)  will  work.  Typically,  some  slack  or  error  variables  were  added  in  the 
formulation  which  provide  convenient  unit  vectors.  At  worst,  with  a  full  matrix, 
an  arbitrary  element  can  be  selected  to  clear  its  column;  any  dependent  columns 
will  show  up  as  columns  of  zeroes  and  cannot  be  used.  Note  that  "frozen"  variables 


o  £  tj  1  0 


(xj  =  '3  ’  0> 


can  conveniently  be  regarded  either  as  basic  variables,  or  as  nonbasic  variables 
at  either  bound. 

Once  the  tableau  has  been  put  in  normal  form,  it  is  probably  most  efficient 
and  convenient  to  leave  it  in  that  form,  making  further  changes  only  by  pivot 
operations.  To  do  this,  the  following  result  is  needed: 

The  optimal  basic  set,  ,  from  one  iteration  of  the  program 
(Bl)  subroutine  may  be  used  as  the  starting  basic  set  for  the 

„ next  iteration. 


The  proof  is  trivial,  once  we  note  (2.9),  the  fact  that  all  are  reset 

to  zero  for  the  next  iteration,  and  the  remark  about  frozen  variables  made  above. 

Suppose  the  objective  coefficients  at  iteration  t  were  tj  (J  ■  1,  ...»  n); 

* 

the  optimal  values  of  the  (tj)  can  be  determined  directly  from  the  pricing  row 
of  the  first  tableau  by  subtracting  { e j }  .  Then,  if  the  same  basic  set,  J  , 
is  kept  to  start  the  (t  +  l)8t  iteration,  one  changes  the  nontableau  rows 


first,  and  then  modify  the  bounds  only  for  the  activities  which  changed  state  in 
the  dual  step  (6.4). 


z 


* 
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APPENDIX  C.  THE  INCREMENTAL  rP.O(,;:  ‘ .  i  )  O';  TIM  SYMMETRIC  PROBLEM 

In  the  symmetric  problem  of  Section  9,  a  negative  unit  matrix  is  in  the 
formulation  explicitly;  by  extending  the  indc  ::  of  from  1  to  m  +  n  ,  we  get 

the  incremental  changes  on  the  {r.l  as  well  ns  the  {x^}  .  Figure  Cl  shows 
how  the  tableau  might  be  organize  i;  might  as  well  be  taken  as  the  last  m 

columns.  Note  that  bounds  and  pricing  nou  apply  to  all  m  +  n  columns;  if  one 

of  the  last  m  columns  is  basic,  a  slack  change  may  occur.  At  optimality,  the 

*  * 

pricing  row  actually  contains  r  +  ,  (k  1,2,  .  . .  ,  n)  or  -rijc_n  +  , 

(k  =  n+l,n+2 . n+m)  • 

If  there  is  a  "slack"  column  in  J)  which  prices  out  incorrectly  at  some 
iteration,  we  see  that  clearing  the  pricing  row  to  zero  in  that  column  will  intro¬ 
duce  price  changes  in  the  first  n  columns,  and  thus  a  nonslack  variable  will 
become  a  candidate  to  enter  «9.  This  step  will  in  fact  be  a  dual-pivot 
as  described  in  the  dual  simplex  algorithm,  Appendix  D.D.  A  more  compact, 
symmetric  form  could  be  obtained  from  a  condensed  tableau  form  of  Cl  with  rather 
complicated  row  and  column-pivot  rules  [2];  however,  rather  than  strive  for 
complete  symmetry,  we  leave  the  procedure  here  in  its  expanded  form,  so  that 
the  (negative)  inverse  of  the  current  basis  is  always  available. 


I 
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APPENDIX  I).  COMPARISON  WITH  OTHER  SIMPLEX  ATCORTTHMS 

To  fully  appreciate  the  relationship  between  the  COMPLEX  method  and 
the  many  special  algorithms  which  arc  in  current  use,  it  is  instructive  to 
explain  the  possible  variants  in  terms  of  the  optimality  diagram. 


A.  The  Primal  Simplex  Algorithm 

In  its  primitive  form,  a  problem  is  formulated  using  unbounded,  nonnegative 
variables;  nonnegative  "slack"  variables  are  added  at  no  cost  as  needed  to  change 
all  constraints  to  equalities.  It  is  assumed  that  a  p rival- feasible  basis, 
of  nonnegative  {x^}  is  known;  the  nonbasic  variables  have  value  x^  *=  0,  j  t 
Dual  variables  are  determined  by  solving: 


(Dl) 

and  thus  we  may  have: 

(D2) 


‘j ' 1  h 


a  «  c 

ij  J 


j  e  Bh 


<  cm 


>  c, 


J  i  B1 


(If  zj  “  cj  ’  we  may  Perturb  the  ,  so  that  corresponds  only  to  our 

set  B  ). 

On  the  simplified  complementary-slackness  diagram  (Figure  Dl)  ,  all  basic 

variables  0  are  on  B  ,  while  nonbasic  variables  are  either  conforming  in  L(  (2) ) , 

or  nonce: i forming  Q)  (in  K  )' with  -  0  . 

An  arbitrary  activity,  e  ,  of  type  (3)  is  selected  to  enter  the  basis, 

and  x  *  £  is  maximized  while  primal-freezing  all  other  nonbasic  variables, 
e  e 

including  the  other  nonconf 'oming  ones ;  horizontal  movement  of  is  unlimited 

until  some  activity  £  in  B  reaches  its  lower  bound  zero,  and  thus  leaves 
the  basis  and  is  considered  as  an  activity  of  type  0  .  As  discussed  in  Section  8B, 
this  means  that  the  incremental  subroutine  consists  of  a  single  pivot  operation; 
the  dual  changing  step  determines  0  so  as  to  make  activity  e  dual-conforming, 


D.3 


since  it  is  "brought  into  the  basis  BP  "  (i.e.,  will  be  used  to  calculate  the 
2^  from  (Dl)).  This  forced  conformity  may  possibly  lead  to  dual  "overshooting," 
some  conforming  nonbasic  activities  become  nonconforming,  and  vice  versa.  (Activity 
t  ,  however,  will  always  move  into  L  .) 

The  single-pivot  procedure  is  repeated  with  the  new  (possibly  enlarged)  set 
of  nonconforming  activities,  until  all  are  made  conforming,  basic  or  not.  At 
each  step,  the  decrease  in  the  primal  functional  C  is  the  shaded  area  shown, 
and  convergence  (except  for  degeneracy)  argued  on  this  fact. 

Sometimes  the  candidate  e  is  not  selected  arbitrarily,  but  is  chosen 
from  among  those  which  are  "highest  above  B  ",  or  "will  give  the  greatest  area" 
on  their  respective  diagrams. 

B.  Artificial  Variables 


Getting  a  primal-feasible  basis  BP  can  be  a  nuisance.  If  a  unit  matrix 
of  appropriate  sign  cannot  be  found  in  (a^)  »  t^ien  nonnegative  artificial  Variables, 
x  ,  are  placed  in  Bp  as  needed.  Since  they  must  ultimately  satisfy  x  -  0  , 

*3  *3 

they  are  given  a  transfinite  cost  M  (Figure  D2a)  ,  but  are  considered  conforming. 

In  this  "method  of  penalties",  solution  of  (Dl)  gives  transfinite  "nonconformities" 
M(Zj  -  Cj)  to  some  of  the  ordinary  variables;  then  application  of  the  primal 
simplex  algorithm  removes  these  artificial  variables,  one  at  a  time,  to  position 
©  in  Figure  D2a  whence  they  are  ignored.  (Assuming  primal  feasibility).  Since 
finite  2^  -  Cj  discrepancies  are  ignored  during  this  procedure,  further  finite 
steps  are  still  needed  to  correct  any  activities  which  are  or  become  finite 


nonconforming. 


z 


z 


Slack  Not  Allowed 
Slack  Allowed 


G' 
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In  the  "two-phase  method,"  transfinite  numbers  are  avoided,  but  a  separate 


cost  functional,  with: 


1  j  artificial 
0  j  ordinary 


is  used  in  "Phase  1"  (Figure  (D2b)).  This  procedure  will  (if  the  problem  is 
primal-feasible)  drive  all  the  x  to  zero.  In  "Phase  II",  these  variables 

3j  n 

are  frozen  at  zero,  and  the  ordinary  functional  C  is  taken  up,  since  a  primal- 
feasible  basis  of  ordinary  activities  is  available. 


As  is  well  known,  the  above  two  methods  are  equivalent. 

In  the  formulation  of  our  algorithm,  these  variables  are  treated  either  as 
error  variables  which  are  to  satisfy  x  =0  (dotted  line  in  Figure  D2c),  or 
as  nonconforming  (negative)  values  of  slack  variables  (solid  line  in  Figure  D2c). 
In  our  approach,  these  would  individually  (or  jointly)  be  made  conforming,  follow¬ 
ing  the  usual  breakpoint  procedure.  To  obtain  the  same  sequence  of  steps  as  the 
above  methods,  however,  one  would  need  to: 


(a)  Take  all  of  these  artificial  variables  as  (part  of)  the 
initial  basic  set  Jj°  ; 

(DA)  (b)  Set  Ej  =  1  for  these  variables  in  the  subroutine  (2.1); 

and  (c)  Ignore  the  nonconformity  of  the  variables  not  in  «9  until  after 

all  these  special  variables  have  been  made  conforming  (i.e.,  until 
the  next  application  of  the  subroutine). 


C.  Combined  Methods 


In  various  synthetic  methods,  it  is  proposed  to  combine  Phase  I  and  II  by 


using 


f(w)  *-  )  x  +  w  J"  c.  x. 

‘i  i  J 
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as  a  single  objective  function,  with  w  as  a  parameter.  Hopefully,  for  some 
finite  w  ,  this  would  eliminate  artificial  variables,  and  still  give  a  solution 
not  too  far  from  the  optimum.  If  not  all  x  =  0,  w  would  be  decreased,  and 

ai 

the  minimization  continued;  in  the  worst  case,  it  might  be  necessary  to  set 
w  •=  0  ,  and  enter  a  true  Phase  I  to  decide  if  the  problem  is  infeasible. 

This  approach  gives  an  obvious  modification  to  subsection  B  above,  or  may 
be  thought  of  as  a  cost-parametric  procedure  (Subsection  (1,  below). 


D.  The  Dual  Simplex  Algorithm 


In  the  dual  simplex  algorithm,  a  dual- feasible  basis ,  B  ,  is  given,  such 
that  when  solving  (Dl)  ,  ve  have 


■j  -  I  Vi 


c\  «  C 

ij  j 


ZJ  1CJ 


j  c  B 


J  t  B°  . 


The  corresponding  primal  solution: 


JD  ViVbi 

XJ  -o 


1  *  If  1 1 « i  in 


j  t  * 


must,  then,  have: 


Some  xj  <  0 


j  e  B 


if  optimal  basis  has  not  been  picked.  On  the  optimality  diagram  of  Figure  D3, 

then  have  dual-feasible  solution  points  which  are:  ©  basic,  primal - feasibl .• 

.);  ©  nonbnsic,  zero-values  (in  I.) ;  or  ©  basic,  primal  infeasible  (  n  B  ).  « 


L 


An  arbitrary  activity,  l  ,  in  B  is  selected  to  leave  the  basis,  (i.e.,  one 
maximizes  £  ),  while  dual-freezing  all  basic  variables.  The  incremental  subroutine 
and  the  dual-changing  steps  (6. A)  (6.5)  together  constitute  a  single-pivot  operation 
in  which  the  basic  variable  is  increased  until  it  is  conforming  (i.e.,  tc^  *  x^) 

to  select  the  vector  to  enter  the  basis,  all  activities  in  L  which  move  towards 
the  right  are  considered  (i.e.,  all  variables  x^  such  that  j  <  0  »  where 

i(i)  is  the  row  corresponding  to  basic  activity  i) .  The  actual  vector  e  is 

» 

conforming  again,  in  state  B  ,  i.e.,  when  it  enters  the  basis  at  the  dual-changing 
step  (6.4)  (6.5).  (See  also  Subsection  8F.) 

There  is  no  difficulty  with  the  other  nonbasic  variables,  since  they  are  not 
selected  to  enter  the  basis,  and  hence  remain  at  level  zero;  however,  if  all 


,  so  that  it  would  be  the  first  to  be 


chosen  to  minimize 


<CJ  -  V'  1  “KO 


a<(o\  *  are  —  0  f°r  some  i(0  •  then  no  candidate  could  enter  the  basis  to 

i  v*-l ,  j 

replace  f,  ,  and  the  primal  problem  would  be  infeasible  (an  £  *  0  cannot 


be  used  as  a  pivot). 
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Forcing  to  leave  the  basis  in  one  pivot  may  cause  "overshooting"  in  the 

sense  that  some  basic  variable  in  B  may  become  primal  infeasible  in  B~  ,  and 
hence  a  future  candidate  to  leave  the  basis;  however,  convergence  is  argued  on  the 
Increase  in  the  dual  functional  B  which  is  equal  to  the  shaded  area  In  Figure 
D3.  As  in  the  primal  simplex  algorithm,  special  rules  of  choice,  6uch  as  the 
"most  nonconforming"  (smallest  x^  <  0),  or  "greatest  shaded  area"  can  be  used. 

E.  The  Artificial  Constraint 

The  dual  simplex  method  is  usually  applied  because  a  dual-feasible  basis  Is 
known,  as  in  parametric  problems.  However,  if  a  basis  B°  is  known  which  is 
not  dual-feasible,  then  an  artificial  constraint 


may  be  added,  with  xq  a  slack  activity.  This  gives  an  optimality  diagram  for 
xq  which  looks  like  Figure  (D4a)  with  an  enlarged  basis  for  the  extended  problem  • 
of  B°  +  {0}  . 


FIGURE  DA:  OPTIMALITY  DIAGRAMS  FOR  DUAL  SIMPLEX  ARTIFICIAL  CONSTRAINTS 


Now  among  the  non-dual-feasible  activities  j ,  j  i  B°  ,  there  is  one  which  is 
"highest"  above  line  B  in  Figure  D4b,  i.e.,  some  k  for  which 
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(DIO) 


2k  "  Ck  "  MaX  t2j  ‘  Ci]  ‘ 
JeB° 


YCj 


By  pivoting  on  x^  in  (D9)  ,  all  of  these  points  are  made  conforming  on  L  , 
with  x^  replacing  xq  in  the  now  dual-feasible  extended  basis 

(Dll)  BD  -  B°  +  {k} 

at  level 

^  -  M  . 

Unless  xq  comes  back  into  the  basis  at  a  positive  level,  the  optimal  solution 
•ay  be  unbounded. 

F.  The  Primal-Dual  Method 

In  the  primal  dual-method  [7],  one  begins  with  a  (not  necessarily  basic) 
dual  feasible  solution  {y^}  ,  and  a  primal  in  which  nonnegative  artificial 
variables  x  have  been  added  to  every  primal  constraint.  This  gives  the 
optimality  diagrams  shown  in  Figure  D5a  and  b. 


FIGURE  D5 :  OPTIMALITY  DIAGRAMS  FOR  THE  PRIMAL-DUAL  METHOD 
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The  Initial  starting  solutions  are  the  nonbasic  variables  *  0  ,  and 
basic  variables  x°  =  b,  (note  that  z°  =  y°)  . 

aj  J  J 

If  z°  <  Cj  ,  then  all  such  variables  are  "frozen"  in  L  ;  only  variables  for 
which  Zj  *=  Cj  are  allowed  to  vary  in  an  associated  restricted  primal.  The 
objective  of  this  general  linear  programming  problem  is  to  minimize  the  error 
terms : 


(D12) 


Minimize  E  =  £  x  , 


while  keeping  all  ordinary  variables  conforming  (in  L  or  B) .  When  E  **  0  , 
the  optimal  solution  is  obtained. 

This  method  can  be  directly  explained  as  a  special  case  of  the  algorithm  of 
Section  6,  when  applied  to  nonnegative  unbounded  variables. 

(a)  The  indices  of  the  artificial  variables  are  taken  as  the  set  <9°  ; 

4" 

these  variables  are  nonconforming  in  L 

(b)  The  ordinary  variables  are  considered  to  be  conforming  in  L  if 
Zj  -  Cj  <  0  ,  or  in  B  if  z^  «  c^  . 

(c)  In  applying  the  incremental  subroutine ,  all  nonconforming  (artificial) 
variables  are  worked  on  simultaneously  (e^  ■  -  1  for  J  nonconforming). 

Otherwise,  the  primal-dual  algorithm  is  identical  to  our  algorithm  with  no 
special  freezing  or  one-pivot  rules  being  used  (except  possibly  between  artificial 
variables  only).  Successive  applications  of  the  incremental  subroutine  will 
reduce  the  artificial  variables,  following  the  usual  breakpoint-stepping  procedure, 
and  the  optimal  solution  to  the  restricted  problem  after  one  pass  can  be  used  as 
the  starting  solution  for  the  next  pass  (the  optimal  J)  from  one  pass  can  be 
used  to  start  the  next  one). 

As  is  well  known,  in  certain  problems  one  may  avoid  putting  in  artificial 
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variables  for  all  constraints;  for  example,  in  flow  problems,  the  conservation 
equations  are  never  violated,  since  each  (incremental)  maximal  flow  subroutine 
only  adds  loop  flows.  Also,  for  this  class  of  problems  the  subroutine  is  solved 
by  a  particularly  efficient  labelling  algorithm. 

If  no  dual-feasible  solution  exists,  then  an  artificial  constraint 


(D13) 

is  added,  and 


n 

y  x  +  X  «=  M 

3-1  3  ° 


(Dl«) 


yQ  »  min  [0,  c^ ] 


Y°  -  0 


i  =  1,  ...»  m 


constitutes  a  dual-feasible  solution  to  the  augmented  program. 


G.  Cost  Parametrization 

If  a  certain  unit  cost,  c^  ,  is  to  be  studied  parametrically ,  starting  from 
an  optimal  solution  (x°  ;  z°)  ,  then  the  substitution 


(D15) 


c3  '  °J  +  8 


is  made  with,  say,  increasing  values  of  0  .  The  optimality  diagram  for  this 
activity  moves  upward  by  an  amount  0  ,  leaving  the  current  solution  (x°;  z°) 
either  conforming  in  L(x°  =0)  ,  or  nonconforming  in  L+(x°  >  0)  .  (Figure  D6) 
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In  the  first  case,  6  can  increase  without  limit,  activity  j  remaining 
nonbasic  in  L  ,  with  no  change  in  C°  -Z>°  . 

In  the  second  case,  j  is  the  only  nonconforming  activity,  so  the  subroutine 
seeks  to  minimize  ,  keeping  all  other  activities  conforming.  After  a  possible 

k 

finite  decrease  in  equal  to  -A^  at  level  c^  ,  a  dual  change  is  made,  moving 

k 

the  current  solution  upwards  by  an  amount  0^  ,  then  to  the  left  by  an  amount 

k 

•••*  etc* •  following  the  familiar  breakpoint  curve.  Finally,  after  a  finite 
number  of  steps,  the  solution  will  be  at  the  desired  level  z^  =  c^  ,  or  x.  =  0  . 
Because  the  starting  solution  was  optimal,  except  for  j  ,  only  one  basis  change 
will  usually  be  made  in  the  subroutine  as  an  activity  in  L  changes  to  B  or 
vice  versa  (except  for  degeneracy). 

/»  * 

In  the  familiar  tracing-out  of  the  piecewise  linear  curve  of  versus 

0  ,  the  vertical  displacements  e1»02’  etc‘  *  corresPond  to  intervals  between 

•  '•  _*  * 
turning  points  of  C  ,  and  the  determination  of  A  corresponds  to  the  basis 

change  at  these  turning  points.  The  decrease  in  &  at  the  origin  0  =  0  is 

-A^  (Cj  -  Zj)  ,  and  the  increase  in  0  when  0  changes  to  0^  is  0.^  (x^  +  A^) 


Complementary  remarks  apply  to  a  decrease  in  0  . 
If  a  vector  change  in  the  c^  is  proposed: 


(D16) 


c]  ■  cj  +  0ej 


j  =  1,2,  •••>  n 


then  the  several  nonconforming  activities  are  worked  upon  at  the  same  time  by 
the  incremental  subroutine  with  ■  -e^  . 


H.  Right-Hand  Side  Parametrization 


First,  suppose  some  requirements  value,  b^  ,  is  to  be  changed  to: 


(D17) 


b±  =  bi  +  $ 


starting  from  an  optimal  solution  (x°  iz°)  .  The  corresponding  equality  -onstraini 


F 
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can  again  be  satisfied  by  adding  an  artificial  variable 


x 

a 


(D18) 


J-l 


aij  Xj  +  Xa  '  bi 


at  level 


This  variable  is  now  nonconforming  with 


o 

z 

a 


o 

y 


as  shown  in  Figure  D7a. 


FIGURE  D7 :  OPTIMALITY  DIAGRAMS  IN  RIGHT-HAND- SIDE  PARAMETRIZATION 
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Since  this  is  the  only  nonconforming  variable,  successive  applications  of  the 

algorithm  will  always  be  minimizing  £  »  and  the  dual  variable  y  will 

fl  1 

increase  as  decreases  (  $  increases).  Horizontal  displacements  correspond 

* 

to  changes  in  C  ,  while  vertical  displacements  cause  changes  in  2)  ,  while 

(y  ;  <f>)  traces  out  the  familiar  breakpoint  curve. 

If  the  right-hand  side  change  is  in  an  inequality, 


(D19) 


XJ  -  k3  +  ♦ 


with  Xj  >  0  ,  then  no  change  will  occur  for  <J>  positive,  j  remaining  in  B 

(Figure  D7b).  However,  for  some  negative  $  ,  J  will  become  nonconforming  in 

B+  ,  and  a  breakpoint  curve  similar  to  the  requirements  change  above  must  be  traced  out. 

Obvious  remarks  apply  to  vector  changes  in  the  {b^}  »  changes  in  some  , 

etc.  (See  also  Section  9.) 


I.  Composite  Methods 

In  the  composite  methods,  such  as  the  self-dual  parametric  algorithm  [5]  , 
or  the  composite  algorithm  [13]  one  permits  a  starting  solution  with  a  basis  B° 
in  which  both 

(D20)  x°  <  0  for  some  (possibly  all)  J  e  B°  ("primal-infeasible") 


(D21)  z°  >  for  some  (possibly  all)  J  t  B°  ("dual-infeasible"). 


By  a  combination  of  the  parametric  methods  of  the  last  two  paragraphs: 

(a)  Setting  x°  -  x°  and  x,  -  0  in  equation  i  corresponding  to 
•  ^  J  J 

the  activity  j  in  (D20) ,  and  then  driving  all  x  to  zero; 

*i 

and  (b)  Increasing  c^  temporarily  to  the  level  z’j  ,  and  then  decreasing 
all  of  them  parametrically  to  zero,  for  all  j  in  (D21). 
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one  can,  it  is  explained,  always  work  with  a  primal-and-dual-feasible  basic 
solution  and  proceed  to  the  optimum  "by  successive  application  of  either  the 
primal  or  dual  simplex  rules."  (Subsection  A  and  C.)  These  parametric  changes 
can  all  be  worked  on  simultaneously,  or  one  at  a  time  (usually  ignoring  the 
others) . 

A  much  simpler  explanation  is  given  in  terms  of  the  optimality  diagram. 


Because  a  basic  solution  is  always  given,  the  nonconforming  activities  are 
either  in  B  or  have  (z°  >  and  x°  =  0)  (Figure  D8) .  Application  of  the 
algorithm  takes  B°  for  «H°  ,  and  always  tries  to  increase  for  all  j  in 

(D20)  and  (D21),  either  one  at  a  time,  or  simultaneously.  Wolfe  [15]  gives  some 


suggestions  for  efficient  choice  of  the 


based  on  computational  experience. 


A  composite  method  is  also  described  by  Balinski  and  Gomory  [2],  which  avoids 


degeneracy  through  a  hierarchy  of  "sub-primal"  and  "sub-dual"  tableaux. 
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APPENDIX  E.  AN  EXAMPLE 

As  an  example  of  the  COMPLEX  approach,  consider  the  following  program 
Min  C  =  2  max  (0,  x^  -  1)  +  2  -  x^ 

X1  +  2  x2  +  x3  -  2 

X1  +  6  x2  +  5  x^  -  -  4 

0  £  Xx  £  3  r  00  £  x2  £  +  2  *  I  I  £  A  . 

We  add  variables  x^  and  x^  to  the  constraints,  together  with  the  bounds 

-  °°  £  _<  0;  0  <_  x^  <_  0  , 

and  take  the  simple  (but  poor)  starting  solutions 
x°  -  (0,  0,  0,  2,  -4) 

y°  -  (0,  0)  (*°  -  0°  -  0  ~  2M  +  4  +  2M  +  4M  »  8M  +  4 

*°  -  (0,  0,  0,  0,  0) 

Only  activity  01  is  initially  conforming.  The  optimality  diagrams  are  shown  in 
Figure  El;  note  that  no  prior  manipulation  of  the  problem  is  needed  to  draw 
these  diagrams. 

The  following  steps  represent  a  possible  sequence  of  iterations  following 
a  COMPLEX  procedure;  the  appropriate  breakpoint  trajectories  are  shown  in  Figure  El. 

Iteration  1: 

Max  Cj  *  J°  ■  (4,5)  .  After  two  zero-level  pivots,  x^  Increases  to  its 

*  *  1 

corner  point,  increasing  x,.  ,  and  decreasing  •  A  ■  2  •  9  ■  y  is  limited 

by  the  downward  movement  x^  •  ■  {2,5}  .  Activity  03  is  now  conforming 
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K*  =  (1,  -  }  ,  0,  0,  2);  =  (2,  0,-  2,  3,  -  1) 

x1  -  (1,  -  |  ,  0,  0,  2);  z1  -  (1,  0,  -  1,  3/2,  -  |) 

y1  -  (3/2,  -  C*  -  ft1  -  6M  . 

Iteration  2: 

* 

Max  .  A  zero-level  pivot  leads  to  A  =  0  and  the  new  basis 
2  *  1 

J)  ■=  {3,  5}  .  0  <=  -j-  is  limited  by  the  upward  movement  of  to  a  corner  point. 

£*  -  (0,  0,  0,  0,  0,  0);  *  (4,  4,  0,  5,  -  1) 

x2  *=  x1  ;  z2  =  (2,  1,  -  1,  -  3/4) 

y2  =  (^-,-  3/4)  ;  £2  -  02  =  5M  . 

Iteration  3: 

.  * 

Max  .  Increasing  x^  drives  activity  v5  conforming,  with  A  ■»  2  .  Since 

*  * 

all  components  of  £  are  zero,  0  may  take  on  any  value,  but  does  not  effect 
£3  .  J  3  *  {1,3}  . 

K*  -  (|,  0,  -  -j,  0,  2);  c*  -  (0,  0,  0,  0,  0) 

•»  11  3  2 

-  (3/2,  -  y,  -  y,  2,  0);  z  -  z* 

y3  -  (^,  -3/4);  £3  -  ft3  -  3M  . 

Iteration  4: 

Min  .  Decreasing  increases  to  its  new  corner  point,  also  de¬ 
creasing  x3-  A*  -  -  6/5  ,  and  -  {3,4)  .  6*  *=  5/4  is  limited  only  by  the 

upward  movement  of  x^  into  a  conforming  state 
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*  3  6  *  (l  A 

K  =  (3/2,  0.  -  Jo  *  "  5  *  0):  C  =  (5*  5*  °*  1*  "  5) 

4  14  4  4 

x  -  (3,  -  j.  "  5.  5.  0)  ;  z  =  (3,  2,  -  1,  4,  -  1) 

y4  «  (4,  -  1);  £4  -  ft4  =  j  M  . 

Iteration  5: 

* 

Min  •  Increasing  decreases  both  x^  and  x^  ,  with  A  «=  -  4/5 

as  activity  //4  becomes  conforming.  =  { 2 ,  3}  .  There  are  no  dual  changes: 

-  (0,  1,  -  6/5,  -  4/5,  0);  C*  -  (0,  0,  0,  0,  0) 

x5  -  (3,  y,  -  2,  0,  0)  ;  z5  -  z4  *=  (3,  2,  -  1,  4,  -  1)  . 

Since  all  states  are  now  conforming,  the  above  solutions  are  optimal,  with 
y'*  *  y4  ■  (4,  -  1)  and  .  ~  *  0  • 

Note,  in  particular,  that  the  trajectories  in  conforming  states  need  not 
be  always  in  the  same  direction;  indeed,  they  may  sometimes  pass  through  the 


same  corner  point  several  times. 
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